ConcreTizer: Model Inversion Attack via Occupancy Classification and Dispersion Control for 3D Point Cloud Restoration
Authors: Youngseok Kim
Sunwook Hwang
Hyung-Sin Kim
Saewoong Bahk
Venue: ICLR 2025 Poster (2025)
Keywords: Inversion attack
3D point cloud data
Autonomous vehicle
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: Inversion Attack via Occupancy Classification and Dispersion Control for 3D Point Cloud Restoration

Abstract: The growing use of 3D point cloud data in autonomous vehicles (AVs) has raised serious privacy concerns, particularly due to the sensitive information that can be extracted from 3D data. While model inversion attacks have been widely studied in the context of 2D data, their application to 3D point clouds remains largely unexplored. To fill this gap, we present the first in-depth study of model inversion attacks aimed at restoring 3D point cloud scenes. Our analysis reveals the unique challenges, the inherent sparsity of 3D point clouds and the ambiguity between empty and non-empty voxels after voxelization, which are further exacerbated by the dispersion of non-empty voxels across feature extractor layers. To address these challenges, we introduce ConcreTizer, a simple yet effective model inversion attack designed specifically for voxel-based 3D point cloud data. ConcreTizer incorporates Voxel Occupancy Classification to distinguish between empty and non-empty voxels and Dispersion-Controlled Supervision to mitigate non-empty voxel dispersion. Extensive experiments on widely used 3D feature extractors and benchmark datasets, such as KITTI and Waymo, demonstrate that ConcreTizer concretely restores the original 3D point cloud scene from disrupted 3D feature data. Our findings highlight both the vulnerability of 3D data to inversion attacks and the urgent need for robust defense strategies.

Published as a conference paper at ICLR 2025
CONCRETIZER:
MODEL INVERSION ATTACK VIA OCCUPANCY CLASSIFICATION AND DISPERSION CONTROL FOR 3D POINT CLOUD RESTORATION
*Sunwook Hwang HHyung-Sin Kim l DDepartment of Electrical and Computer Engineering, Seoul National University
Youngseok Kim!
iSaewoong Bahk
2 Systemm LSI, Samsung Electronics
3 'Graduate School Data Science, Scoul National University
yskim@netlab. anu.labbauuu...... ac kr, sunw sun..wwanne www.ge ams com, (byungkim, sbahk) assu.ac....
ABSTRACT
The growing use of 3D point cloud data autonomous vehicles (AVs) has raised serious privacy concerns, particularly due to the sensitive information that can be extracted from 3D data. While model inversion attacks have been widely studied in the context of 2D data, their application unexplored. To fill this gap, we present the first in-depth study of model inversion attacks aimed restoring 3D point cloud scenes. Our analysis reveals the unique challenges, the inherent sparsity of 3D point clouds and the ambiguity between empty and non-empty voxels after voxelization, which are further exacerbated by the dispersion of non-empty voxels across feature extractor layers. To address these challenges, we introduce Concre Tizer, simple yet effective model inversion attack designed specifically for voxel-based 3D point cloud data. Concre Tizer incorporates Voxel Occupancy Classification to distinguish between empty and non-empty voxels and Dispersion- Dispersion-Controlled Supervision to mitigate non-empty voxel dispersion. Extensive experiments on widely used 3D feature extractors and benchmark datasets, such as KITTI and Waymo, demonstrate that ConcreTizer concretely restores the original 3D point cloud scene from disrupted 3D feature data. Our findings highlight both the vulnerability of 3D data to inversion attacks and the urgent need for robust defense strategies.
3D point clouds remains largely
INTRODUCTION
Recent advancements in Autonomous Vehicles (AVs) have underscored the importance continuous vision data collection and sharing. At the same time, the widespread adoption of AI technology has amplified privacy concerns, prompting increased research on this issue (Guo et al. 2017 Stahl & Wright 2018). Consequently. AV's data collection faces strict regulations that requires data de- identification (Mulder & Vellinga 2021). For example, the EU's General Data Protection Regulation (GDPR) EU 2016) mandates businesses to adopt stringent data protection protocols.
Beyond these regulations. the need for privacy preservation is rapidly increasing, particularly in 3D point cloud data. This is because various types of privacy-related information can be revealed through rich 3D shape information. For instance, personal identities can be exposed through facial recognition (Zhang et al. 2019) and person re-identification (Cheng & Liu 202T). Additionally, behavioral patterns can be inferred from human pose estimation (Zhou et al.) 2020) and activity recognition (Singh et al., (2019)). Location information can also be extracted using techniques like Simultaneous Localization and Mapping (SLAM) (Kim et al. 2018). Furthermore, the ability to
reconstruct 2D images from sparse 3D data (Pittaluga et al. 2019 Song et al. 2020) emphasizes the importance of securing raw 3D point data from the outset.
Both authors contributed equally to this research.
This work conducted while the author was affiliated with Scoul National University.
t Corresponding authors.

---

Published as a conference paper at ICLR 2025
Original data
Feature data
Restored data
Entire Scene
Object (Vehicle)
Feature
Conventional Inversion Attach
Generative model
appreach
ConcreTizer
Extraction
Figure 1: Inversion attack results of 3D point cloud. Feature data is extracted from original point cloud through a 3D feature extractor Yan et al. (2018). Concre Tizer (right) enables restoration with simple modifications to conventional approach (left), and even achieves more concrete restoration than generative model approach (middle) (Xiong et al. 2023
Downserpling
Upsimpling
Original data
Restored data
Cmmpte rool
MSE loss
Figure 2: Restoration through conventional inversion attack method. Voxelization introduces zero-padding to empty voxels. During downsampling and upsampling, non-empty voxels spread to neighboring areas, expanding the Vol (green region). Within the Vol, voxel-wise channel regression generates additional points in sero-padded regions, leading to clustering near the origin.
However, research on privacy in 3D point cloud data remains significantly underexplored compared advancements in the 2D image domain. A prominent research area in 2D image privacy is inversion attack, which aims restore the original data from extracted feature. While earlier studies (Gupta & Raskar 2018; Vepakomma et al. 2018 Singh et al.) (019a) suggested that 2D images could be anonymized by extracting features, inversion attacks have demonstrated that these features can be used to restore the original 2D images. In contrast, while there have been a few prior studies on privacy of 3D data (Wang et al. [0244),, inversion attacks on 3D data remain largely unexplored. This research gap allowed recent study (Hwang et al.) 2023) to operate under the assumption that disseminating 3D features inherently prevents the restoration the original data. In the absence of existing inversion attack methods for 3D data, the authors developed a Point Regression method to invert voxel- based backbones, aiming to demonstrate that restoring the original 3D scene from its extracted features is infeasible. As in Figure D. the conventional Point Regression in (Hwang et al.) 2023) fails to restore 3D point cloud data from intermediate features.
We argue that this failure is not due design that considers the characteristics of 3D backbones. To address this issue, Figure eigureeex examines the phenomena arising when the Point Regression method inverts voxel-based feature extractors, which are dominant architectures in autonomous driving applications. The Point Regression approach attempts directly restore point coordinates within each voxel by minimizing mean squared error (MSE). The problem is as follows: The sparsity of 3D point cloud data results in large number of rero-padded voxels. To identify the meaningful regions within the voxel grid, we define Vol (Voxels-of- Intereest) as the set of non-empty voxels, which contain valuable information. During both feature extraction and inversion processes, Vol spread into empty voxels. This dispersion leads to proliferation of false Vol (originally empty voxels), causing Point Regression to erroneously generate points in regions that were initially void. Moreover, these false Vol disproportionately impact the MSE loss, prompting the Point Regression model to bias the restoration by concentrating most points near the origin (0, O, 0) to minimize estimation errors for the false Vol. This bias significantly degrades localization performance for the relatively smaller number of true Vol (originally non-empty voxels).
inherent safety of 3D features but rather a lack of careful
2

---

Published as a conference paper at ICLR 2025
The analysis reveals that the key successful inversion attack is not restoring the representation of the voxel (i.e., point coordinates) but accurately determining whether a voxel was originally empty or non-empty. Once this classification is achieved. localizing points within non-empty voxels becomes more straightforward, as the error is constrained by the typically small voxel size. Based on this insight, transform the conventional Point Regression problem into a more explicit Voxel Occupancy Classification (VOC) problem. In addition, the spread of Vol should be suppressed during restoration to minimize the negative impact of false Vol. To address this, our model incorporates Dispersion- Controlled Supervision (DCS), which segments the feature extractor based on downsampling layers and trains each segment individually, proactively controlling the dispersion of Vol. Thanks to its tailored design, our model, Concre Tizer, even outperforms the generative model approach that uses conditional generation (see Figure
[ the generative model approach (Xiong et al.) 2023)).
To demonstrate the general applicability of ConcreTizer, we deployed it on two representative 3D) feature extractors (Yan et al., 2018 Lu et al. 2022). which are essential components in various applications including 3D object detection, 3D semantic segmentation, and tracking. Our experiments on the widely used KITTI (Geiger et al. 2012) and Waymo (Sun et al. 2020) datasets confirm that Concre Tizer consistently outperforms across various datasets and 3D feature extractors. We showcase the superior performance of Concre Tizer through a comprehensive set of quantitative and qualitative evaluations, including point cloud similarity metries, visual analysis, task-specific performance (3I) object detection) using restored scenes, and the effectiveness of potential defense mechanisms.
The contributions of this paper are as follows:
This is the first in-depth study on model inversion attacks for restoring voxel-based 3D point cloud scenes, identifying unique challenges from the interaction between sparse point clouds and voxel-based feature extractors.
To address the identified challenges, we propose Concre Tizer, tailored for inverting 3D backbone networks, with Voxel Occupancy Classification and Dispersion- Supervision.
Through extensive experiments with representative 3D feature extractors and well-established open-source datasets, we demonstrate the effectiveness of ConcreTizer both quantitative and qualitative aspects.
RELATED WORK
3D Point Clouds Feature Extraction. Feature extractors for 3D point cloud data encompass set, graph, and grid-based approaches, each distinguished by its representation format. The computational
complexity of set and graph-based methods (Qi et al. 2017 Kipf & Welling 2016 Park et al.
2023) scales significantly with the number of points, limiting their use real-time applications like autonomous driving. Conversely, grid- based methods (Zhou & Tuzel 2018 Yan et al. 2018 Shi et al.
2020 Sun et al., (022) organize the 3D space into voxel grid and apply specialized convolution (Liu et al. 2015) Graham & Van der Maaten 2017) for efficient feature extraction from sparse data. This efficiency makes them particularly well-suited for autonomous driving applications. Based on these characteristics, we investigate inversion attacks for scenarios using voxel-based feature extractors.
Model Inversion. Model inversion was originally explored in the context of interpreting deep learning models. Traditional approaches generate saliency maps to understand how models produce outputs (Du et al. (018). Other methods (Mahendran & Vedaldi 2015: Dosovitskiy & Brox 201 a) reconstruct the input from intermediate features to analyze the information flow through model layers. Recently, with growing concerns about data privacy, model inversion has gained attention as privacy attack. Early studies attempted to restore input face images from confidence scores (Yang et al..
2019b). Subsequent studies (Zhang al. 2020 Zhao et al.
2021) leverage additional information for more sophisticated restoration. Building on these studies, corresponding defense techniques (Liu et al. 2019 Xue et al.) 2023 Dusmanu et al.) 2021 Ng et al. 2022 Zhang et al.) (022) have also been investigated, enriching the exploration of data privacy. However, existing work has primarily focused 2D image data. There is clear need for inversion attack technique that accounts for the unique characteristics of 3D point cloud data in autonomous driving. To the best of our knowledge, this research the first to study inversion attacks on 3D data.
Point Cloud Generation. Generative models widely used, owing to their diverse range of applications. In the 3D point cloud domain, several generative models are actively being explored. Unconditional generation tasks aim create plausible 3D shapes from random inputs, such as
3

---

Published as a conference paper at ICLR 2025
Original
Coveoopann Frogge
16.37 polt
Vostlattzn
Dewrsamphing
Upsampiing
Reamity
Dramsampingg
Upsaroiing
Conventional invenves otteck
3gg foyer
Fig fnyer
9, Agyer
12g tayer MENM pa
Dud puints
points
Figure 3: (Left) The results of the conventional inversion attack: number of restored points increases rapidly, and the concentration of points the origin becomes more noticeable. (Right) The Vol (Voxels-of-Inteee) spread as they pass through the feature extractor and inversion attack model.
As the layer depth increases, the
dispersion effect: The non-empty voxels
noise (Achlioptas al. 2018 Valsesia et al. 2018 Yang et al. 2019a Luo & Hu 202 Conditional
generation tasks involve generating the missing part of a point cloud Yu et al. 2021 Huang et al.
2020: Wen et al. (2200) producing 3D point cloud from 2D image (Mandikal et al. 2018
et al. 2023). However, most existing research focuses on dense point cloud data for individual objects (e.g.. Chang al. (2015)). Only few studies (Caccia et al. 2019 Zyrianov et al. 2022) deal with scene-level sparse point clouds captured from autonomous vehicles. Even these studies require specific representation formats and do not support using 3D grid-type features, as conditions in our inversion attack scenario. To our knowledge. the only scene-level sparse point cloud generation model based on 3D grid representations Xiong et al. (023). We also conducted performance comparisons with conditional generation approach.
Mandikal & Radhakrishnan 2019: Melas-
PRELIMINARY: LIMITATIONS OF CONVENTIONAL INVERSION ATTACK
The only known attempt at inversion attack 3D point cloud data is by Hwang et al. Even this research does not directly focus on inversion attacks but rather seeks to assess the privacy protection effectiveness of 3D features by developing simple inversion attack based on Point Regression. Before designing our method, we explore why the conventional approach can not effectively restore 3d point cloud scenes (Figure left).
Firstly, we identified an issue in voxel-based models related to the voxelization process. During voxelization, regions without points are zero- pero-padded. However, conventional regression method does not consider point existence but focus solely on point localization, mistakenly interpreting zero-padded representations as valid points located at (0, O, 0). As a result, points are created even for empty voxels, leading to an overgeneration of points compared the original data. Secondly, the inherently sparse nature of point clouds results in a large number of zero sero-padded voxels, far exceeding those containing valid points. Since Point eagression-based inversion attacks aim to minimize estimation errors across all voxels, they unintentionally prioritize zero-padded regions. Consequently, this bias towards zero-padded voxels causes over-concentration of points near the origin in the restored scene. Moreover, it significantly increases localization errors for the relatively smaller number of valid points, these errors become negligible within the overall regression error.
Lastly, we observed that as the feature extractor layers deepen, existing attack methods are increasingly hindered by the negative impact of zero- parded voxels: (1) an excessive number of restored points and (2) an intensified concentration of points near the origin. Specifically, if voxels with a value of (0, 0) persist in the final restored state, they are excluded from the regression targets and do not directly affect the regression loss. However, due to the nature of convolution operations, the values of non- empty voxels-defined as Vol (Voxels-of- Ine---etteet-gdduueeee voxels. As the layers deepen, more originally zero-padded voxels become non-empty during the feature extraction and inversion processes. Consequently, an increasing number these previously zero-pa zero-padded voxels are included in the regression targets. Our experiments revealed that the density of Vol spikes significantly at downsampling layers (Figure right), further amplifying the influence of zero-padded voxels the final restoration results.
disperse into the surrounding empty
4

---

Published as a conference paper at ICLR 2025
Original point dloud
Vooelization result
Vosal accupancy score
Restoration moslt
Interediatt
catrecto (roan))
Tv
model
Attocker Side
Concre Tizer Attacker Scenario
ConcreTizer Training Operation
Occupancy Ciasofication
Dispersion Supervision
Aboat
Brane)
Decupeed
Partiiniig
Emppyy
Abock
locs
0203 010zu4
Q
oinket
57 kra
Proze50200
Q
Mergin
fd
fa
Tax Alaki www.h Slick
Figure 4: ConcreTize framework. Original point cloud and features are denoted as and fi with restored versions as p' and ft respectively, where i indicates the i-th downsampling layer. Concre Tizer restores data by classifying fo occupancy and placing points at voxel centers. For deeper layers, it partitions at downsampling layers to restore fi-1 from ft.
PROPOSED METHOD
4.1 AV SCENARIO
We focus on autonomous vehicle (AV) scenarios due to their high risk of exposure to inversion attacks. In AV contexts, feature data would be shared for purposes such as computation offloading (Xiao) et al. 2022 Hanyao et al. 2021), model enhancement (Hwang al. 2023), and cooperative inference (Wang et al. 2020 Xu et al. 2022 Yu et al., 2022). Specifically, we selected voxel-based feature extractors, which well-suited for real-time processing in AV. Their efficiency makes them essential for tasks such as 3D object detection Yaa et al. 2018 Lang et al. 2019 Shi et al. 2019 2020: Shi & Rajkumar 2020), semantic segmentation (Wu et al. 20119] tracking (Yin et al. (021). In this scenario, an attacker with access to the same feature extractor can easily prepare 3D point cloud data for training the inversion attack model. Since the restoration task doesn't I require separate labeling, they can utilize open-source datasets or self-collected data.
(oo1)TThmmaa Thomas al. 2019), and
4.2 PROBLEM DEFINITION
The goal of an inversion attack to discover the inverse process given feature extractor order to restore the original data. For voxel- base- feature extractors, the initial step involves voxelization process that transforms point cloud data into a grid format. Voxelization converts a 3D point cloud R where & is the number of points, into voxel grid fo 3 where H, W, D represent the spatial dimensions the grid. The x, y. and coordinate information is organized into separate channels, and voxels without points are zero-padded, resulting in channel values of (0, 0). In particular, during the downsampling process, the spatial dimensions shrink while the channel size increases, producing features JN layers, Cy > 3, and hv, uN, dy are smaller than H, W, D. Consequently, our inversion attack aims restore the original voxel grid fo from the downsampled features fN.
R CxCxxx xhx x xdx where N is the number of downsampling
4.3 CONCRETIZER FRAMEWORK
Figure depicts the overall ConcreTizer framework incorporating the scenario and attacker-side training operations. For the design of the inversion attack model, we adopted symmetrical structure to the feature extractor, following previous studies (Yang al. 2019b: Zhang et al. 2020 Zhao et al. (202T). In this approach, the original shape is restored by upsampling at the positions where downsampling occurred (detailed structure is provided in the supplementary material). Building upon symmetric structure, Concre Tizer applies Voxel Occupancy Classification (VOC) and Dispersion- Controlled Supervision (DCS) to overcome the limitations of traditional inversion attack. VOC converts the regression problem into classification problem to address the issue of point clustering near the origin. DCS prevents the dispersion of Vol by splitting the feature extractor, helping to mitigate the degradation of restoration performance as the network deepens.
5

---

Published as a conference paper at ICLR 2025
4.3.1 VOXEL OCCUPANCY CLASSIHICATION
In traditional inversion attack methods, the original data is directly restored through regression channel values. In our scenario, since the x, y, and z coordinates are channelized during the voxeliza- tion process, performing regression would restore coordinate values. However, since voxelization of sparse point clouds produces a large number of pero-padded voxels with (0, O, 0) channel value, many unnecessary points cluster near the origin in the inversion attack results (Figure [ left). To address this issue, we transform the regression problem into a classification problem to resolve the semantic ambiguity of zero-padded woxels-whether they represent empty voxels or valid points at coordinates (0, O, 0). This can be achieved through simple binary encoding, where each voxel is labeled as 0 (negative occupancy) or (positive occupancy). making the meaning of zero-pade pero-padding clear. Using the VOC method, the inversion attack model outputs binary classification scores in the form of I x xD, rather than continuous coordinate values in the form of R xD. If voxel is determined to contain point, the corresponding coordinate can be restored casily. This is because the range of coordinate values is bounded by the spatial location of the voxel, and the voxel size is typically small enough, As result, by using the center coordinates of the voxel, we achieve effective restoration within an error range constrained by the voxel size.
Additionally, due to the sparsity of original point cloud data, the binary- encoded labels contain higher ratio of Os compared to 1s. This phenomenon is particularly exacerbated as the depth of the layers increases. Let So be the original voxelized point cloud and S be the restored one by the inversion attack. The number of positive labels is fixed as Voo of fol. while the number of negative labels, Vol of fól - |Vol of fol. increases exponentially as the depth of the layer increases. To account for this imbalance, we apply the Sigmoid Focal (SF) loss (Lin et al. (217)). a variant of the conventional cross-entropy loss. The mathematical representation of the SF loss is given by (P():) = - (1 pt)" 7 log(Pt), where pt denotes the model's predicted probability for the target class. The factor a₂ is employed to adjust the importance given to the positive and negative classes.
4.3.2 DISPERSION- CONTROLLEE SUPERVISION
While applying SF loss in VOC can partially address the label imbalance issue, it cannot prevent the more inherent problem of Vol dispersion. The original data is sparse with many empty voxels, yet as observed earlier, the Vol density increases exponentially during the downsampling process (Figure 3 right). As the Vol spreads excessively in the deeper layers, it becomes increasingly difficult to restore the data to its original sparse state.
Our proposed DCS offers a more fundamental solution to address Vol dispersion. It divides the feature extractor into multiple blocks and performs restoration progressively. First, the feature extractor is partitioned based on the downsampling layer, where Vol dispersion occurs. In the inversion attack model, a corresponding inversion block is created for each block of feature extractor. This allows the restoration process to be trained in block units, effectively controlling Vol dispersion within each block. It is important to note that, at the original voxel level, the channel values directly represent point coordinates, eliminating the need for regression (if the classification result is positive, the channel value is estimated as the center coordinate of the voxel). However, at the intermediate feature level, normalization applied, which disrupts the direct relationship between the channel values and the voxel location. As a result, both classification and regression on the channel values are required.
For example, if the input to the (i + 1)-th block is S RCxxhixw,x xd, and the output is ʃ1+1 C xh; Xw:; x1 then the (i + 1)-th inversion block in the inversion attack model takes f(+1 as input and produces f/ C. xw; x=;xd; which is the result of restoring ft. Specifically, m/ 1x, xd, (spatial occupancy scores found by applying SF loss) and de E Cx x xd, (channel values found by applying L2 loss) are derived from fi+1. Then, cf is masked by using m² to generate f/. During the masking process, unnecessary voxel values are erased, helping to suppress the dispersion of Vol. Note that in the first inversion block, which is the final stage of the inversion attack model, only classification is performed, with no additional regression. The loss function for each inversion block is:
Lots
if 0,
Losss block + 1) = Lets + +.B Treg
if 2 1.
Legs =
Σ Vol
SF loss (m,, m²) and Leeg = Σ
L22 loss cf)
Vol
6

---

Published as a conference paper at ICLR 2025
Table Inversion attack result with KITTI and Waymo dataset. Average CD and HD values in centimeters, and F1 scores with 15 cm and 30 cm thresholds for KITTI and Waymo datasets. Metrics evaluate over each dataset with 3769 and 3999 scenes, respectively.
(Dewnssmpling
(3rd)
(6th)
(9th)
4
(LayerDDepth)
CT1
CD
(1)
CD1
Wayma ma KITT
Paint Regression
1.3888
253555
0.3543
1.2579
34,2515
0.391
3.1229
54,0173
0.210
4.449
56,9811
0.1298
UltraLiAAR
0.744
0.9122
0,0818
3.0974
0.8905
0.0835
79561
0.8829
0.3012
7,9185
Point Regression
1979
55,589
0.7644
2.723
66. 76.9999
0.489
4,1053
200008
0.522
4.2340
71.9628
0.4355
UDAR
00810
10,9583
0.9742
0,0888
11 11,3500
9633
0.1017
11.4987
0.9503
0.1378
12,0259
0.8849
ConcreTuze Tizer
0.334
10,3544
0,9984
10.2326
0.9979
0,0712
10.5734
0
0.1087
11.3999
0.9251
KITTI Scene
Waymo Scene
Hightihh
Highigh
Tox Anper
G. laper
Du Atper
12g laper
Highight
Sulayeer
4. lager
Oa lejer
12, loyer
Highig
7ar
Contretter
Concretite
Aasoovtian
Figure 5: Qualitative results for KITTI (scene 73) and Waymo (scene 79). Top shows the original point cloud, 2D image, and highlighted region. Below, restoration performance of three techniques is displayed, progressing left to right by layer depth.
Here, m, and m represent the ground truth and predicted spatial occupancy masks, respectively, while c and cf denote the ground truth and predicted channel values. The final result of passing through all inversion blocks is set of binary classification scores in the form of R I fxfxxx> xxxxx Restoration is completed by generating a point at the center of the voxel corresponding to positive occupancy.
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
3D Feature Extractor. We employ doxelization-base 3D feature extractors as the target of our inversion attack. Based on the OpenPCDer (Team) 2020)) project, utilize pre-trained VoxelBack- bone ( Yan et al., 2018) and VoxehResBackhone (Lu et al. (202),, extensively used in key applications for 3D point cloud data. The VoxeIBackBone structure includes four downsampling layers (i.c.. N = 4), each preceded by two convolutional layers, while the VoxeIResBackbone incorporates additional convolutional layers and skip connections.
Inversion Model Training. We train the inversion attack model on the real-world KITTI (Geiger et al. 2012) and Waymo (Sun et al. (2200) datasets. In VOC, when applying the SF loss function, only a in the hyperparameters is adjusted. In DCS, the weight on the regression loss, 3, is set to 1.
Metries. evaluate 3D scene restoration performance, we employ various metries. For qualitative analysis, we visualize the 3D point cloud using the KITTI viewer web tool. For quantitative analysis, we utilize point cloud similarity metrics such as Chamfer Distance (CD) (Borgefors 1984). Hausdorff
Distance (HD) (Huttenlocher al. 1993), and F1 Score (Goutte & Gaussier 2005). Additionally, to assess the utility of the restored data, we examine 3D object detection accuracy using pre-trained detection models.
5.2 RESTORATION PERFORMANCE
Comparison Schemes. To demonstrate the superiority of ConcreTizer, we compare it with two approaches: traditional inversion attack method and generative model-based approach. First, we examine Point Regression (Mahendran & Vedaldi, 2015); Dosovitskiy & Brox (0166 conventional inversion attack method. In this approach, the goal to directly recover the channel
7

---

Published as a conference paper at ICLR 2025
Oigioaa 2212
10
10"
Score
#Ussmemmpiing
(1210))
[I (Layrelhepsh)
Placer (1)
AP
Ponx REgresaco
3:331
35.419
Ulima
7,8521
ConcreTTww
0.0714
IL9350
63.07
Figure 6: Restoration result on BoxeoResBackoone with KITTI dataset. At the left, the last layer's restoration performance for three techniques shown. At the right, average performance across the KITTI dataset is presented. A bar graph depicts relative performance, and table details raw values.
values. To improve the results, we additionally apply post-processing to remove points that fall outside the defined point cloud range or cluster excessively near the origin. Next, we compare ConcreTizer with a generative model-based approach. Inversion attacks using generative models require conditional generation, where feature data serve the condition. Among existing LiDAR point cloud generation models, UltrLiiARR (Xiong et al.] (023) is the only one utilizing a voxel representation similar to feature extractor. To adapt UltrLiiDAR for inversion, modified its encoder to accept voxel features as input.
Result Analysis. Table presents the point VoxeIBack Bone (Yan et al., 2018). while Figure 5 visualizes the corresponding restored point cloud scenes. It evident that Concre Tizer consistently demonstrates outstanding performance across all cases in both the KITTI and Waymo datasets.
restoration performance at different layer depths of
Traditional Point Regression methods prove ineffective for inversion attacks on 3D features. In particular, many points cluster near the origin, and this phenomenon becomes more pronounced deeper layers. This limitation stems from the failure to account for the characteristies of 3D sparse features. By leveraging conditional generation, Ultral .iDAR can restore the overall scene in coarse-grained manner, showing less performance degradation in terms the HD metric as layer depth increases. This rough recovery can be attributed the transformation of 3D sparse features into 2D dense features, which aligns with the 2D VQ- VAE design. Since Vol dispersion no longer present in 2D dense features, UltraLiDAR achieves better stability. However, this conversion leads to the loss of 3D sparse characteristics, resulting in less accurate restoration fine details. contrast, Concre Tizer effectively suppresses Vol dispersion through DCS while preserving the sparse nature of 3D features. Despite its simple design, it achieves more concrete restoration compared to the generative model-based approach. At the deepest layer, Concre Tizer outperforms the generative approach by 23.4% and 12.4% on KITTI, and by 21.1% % and 4.5% on Waymo in terms of CD and FI score, respectilly..
Additionally, Figure presents results for VoxeIResBackkone (Lu al. 2022). When analyzing the representative results from the deepest layer, Concre Tizer exhibits the best performance in CD, F1 Score, and AP3D. A persistent limitation of UltraLiDAR is the lack of detailed shape in the inversion attack result. Detailed experimental results, including those for VoxelResBackbone and the Waymo dataset, are provided in the supplementary materials.
5.3 ATTACK PERFORMANCE THE CONTEXT 3D OBIECT DETECTION
To assess the effectiveness of inversion attack results in terms of privacy compromise, we measure the 3D object detection accuracy using restored point cloud scenes with pre-trained object detection models. Table 2 summarizes the benchmark results for the KITTI and Waymo datasets. Point Regression fails perform inversion attack, producing completely unusable results. UltraLiDAR performs relatively well on KITTI but exhibited poor performance on Waymo, which has a broader range and higher scene complexity. This suggests that while generative models can restore overall scene, they struggle to capture detailed shape. In contrast, only Concre Tizer demonstrates consistent performance across both datasets, achieving 75.5 to 87.0% and 62.6 to 75.7% of the detection performance compared to the original scenes in KITTI and Waymo, respectively.
8

---

Published as a conference paper at ICLR 2025
Table 2: 3D object detection results with KITTI and Waymo datasets. The reported metric for the KITTI dataset is Average Precision (AP) at hard difficulty, while for the Waymo dataset, Average Precision weighted by Heading (APH) is reported LEVEL2 difficulty.
Detection Model
PointPile
PORENN
VoueRCNN PiinRRNN
Danectinn Model
Poinshilbr
PYRONN
VovelRCNN
ConterPoim
Chiginal Uat
16.11
78.82
78.78
78.25
Original Dara
0.5004
0,0034
0,6534
0.6299
Poeni Regression
0
0
0
Poin Regression
0
UltaLiDAR
56.05
39.00
54.19
0.1602
ConsleTizor
39.45
64.27
65.03
Canore
0.2445
0,4399
0,1000
KITTI Scuns
Voc (SCE
KITTI
CononTow
1.0
10"
Store
Wiy
B
Chnorii 100
0.2
Layer Depth
Figure 7: Ablation study on VoxelBackbone with KITTI dataset. At the left, the restoration performance for three cases is shown. At the right, average performance across the KITTI dataset is presented wih boxplot.
5.4 ABLATION STUDY: COMPONENT-WIST ANALYSIS
To understand the performance of ConcreTizer, we analyze the impact of each component. Figure 7 compares the performance of VOC (BCE loss), VOC, and Concre Tizer (VOC+DCS), Firstly, VOC (BCE loss) shows that transitioning from regression to classification, which clarifies the meaning of zero-padded voxels, enables restoration of 3D sparse data (6 layer result). However, BCE loss struggles with significant label imbalance as layer depth increases. Comparing VOC (BCE loss) with VOC highlights that SF loss helps alleviate the label imbalance issuet. Nonetheless, in VOC, the restored points cluster in specific areas, leading to biased restoration (12th. layer result). Only Concre successfully restores points in a distribution closely matching the original data. This success stems from DCS's ' ability to effectively mitigate Vol dispersion, especially in deeper layers.
5.5 PARTITIONING POLICY IN DISPERSION-CONTROLIIC CONTROLLED SUPERVISION
We conduct experiments to identify the effective strategy for applying DCS in Concre Tizer, given specific 3D feature extractor. Restoration performance is evaluated on the KITTI dataset by varying the number of DCS instances (i.e., the number of inversion blocks). In cach case, partitioning is applied at positions that aim to achieve an even division of the total number of layers. As shown in Figure 8 applying 10 DCS instances results in significantly worse performance than not using DCS at all (i.e., DCS 1). This is because the restoration error accumulates as it passes through multiple inversion blocks. The best performance is achieved with 2, 3, or 4 DCS instances, where each partitioned block contains at least one downsampling layer. This can be attributed to the additional supervision effectively suppressing Vol dispersion that occurs during the downsampling process. Therefore, to maximize the benefits of supervision, partitioning should be aligned with the downsampling layers, where Vol dispersion manifests. Qualitative results for different DCS instances and further discussion on the optimal DCS split position are provided in the supplementary materials.
5.6 TRADEOFF BETWEEN PRIVACY AND UTILITY
To analyze the trade-off between utility (3D object detection accuracy) and privacy protection (restoration error), we examine various data perturbation techniques (Wang et al., 2024b Li et al. 2021: Wang et al. 2024a) as potential defense mechanisms against the Concre Tizer inversion attack. We explored two types of perturbations: point cloud augmentations and Gaussian noise addition. For point cloud augmentations,
apply random rotations, random scaling, and random sampling. For Gaussian noise addition, we introduce noise at the feature data level with three region- configurations: distributed noise, which is uniformly applied across all feature data feature centric noise, which is applied only to Vol (regions containing information); and empty-centric noise, which exclusively targeted empty regions.
feature-

---

Published as a conference paper at ICLR 2025
8 9.3
02
6.1
0.8
Number DCS instances
Figure 8: Effect of the number of DCS instances. DCS 1 is the end-to-end approach without
partitioning. DCS 2, 3, and use downsampling- based partitioning. DCS 10 partitions at every layer.
Polist
Table 3: Effect of point cloud augmen- tation. Measured: AD3D (detection ac- euracy) of the SECOND model and CD (restoration error) Concre Tizer.
Rotation (1)
0
2
250
10g
1700
00076
0112
0.728
02310
12548
Scaling
03
01
26
16
CD
AP
00006
81.77
01163
3412
02233
2..0
02790
11.47
03179
8,3
0.3516
Distributed Noise
(b) Feature centric Noise
(c) Emoty centric Noise
Figure 9: Effect of Gaussian noise. Measured: AP3D (detection accuracy) of the SECOND model and CD) (restoration error) of Concre Tizer.
Samplin
AP
61.5
5832
4:31
14.09
C0
01516
0.00
As shown in 3 and Figure 9 these perturbations effectively reduce the restoration capability of the attack (defense) but also degrade object detection performance (target task), highlighting the challenge of mitigating Concre Tizer attacks without significantly compromising system utility. Notably, Figure Irreaals that the sparse nature of 3D feature data causes noise to affect different regions unevenly, emphasizing the importance of considering spatial characteristics when designing future defense mechanisms. More visualization results are provided in the supplementary materials.
DISCUSSION: LIMITATIONS AND FUTURE DEFENSE STRATEGIES
Our inversion attack method demonstrates that 3D features not inherently secure, they can be exploited to restore the original data. This restored data could reveal private information, including personal identities, behavioral patterns, and location details, thereby posing a risk to the applications of voxel-based 3D vision models. Since Concre Tizer assumes that the parameters of the feature extractor are known, protecting model parameters can prevent such attacks. If sharing parameters is unavoidable, defense can be achieved by sacrificing some utility (accuracy of the vision model), as shown in Section55.6) However, in accuracy-critical environments like autonomous driving, simple defense techniques are likely to be inadequate. Future research should focus on developing defenses that mitigate attacks while minimizing the impact on utility. Additionally, for latency-sensitive systems, it is crucial that defenses do not impose significant computational overhead. Potential strategies include Differential Privacy (DP) (Abadi et al.) (016),, Adversarial Training (Liuu al., 2019), and Feature Obfuscation (Zhang al. (022). The strengths and limitations of each technique are discussed the supplementary material.
CONCLUSION
This paper presents the first comprehensive study on model inversion for 3D point cloud restoration. In the context of autonomous driving, we focus on the most dominant voxel-based feature extractors and examine the challenges arising from their interaction with 3D point cloud characteristics. Based on this, we introduce ConcreTizer, a simple yet effective inversion technique tailored for restoring 3D point data from features, which incorporates Voxel Occupancy Classification and Dispersion- Controlled Supervision. Through rigorous evaluations using prominent open-source datasets such as KITTI and Waymo, along with representative 3D feature extractors, we not only demonstrate the superiority of Concre Tizer but also analyze each of its components in detail for valuable insights. Our research reveals the vulnerability of 3D point cloud data to inversion attacks, emphasizing the urgent need devise extensive defense strategies. While this work focuses voxel- representations, we see inversions attacks for more diverse representations of 3D data, such as point set and graph, as valuable future work.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported in part by Institute Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. LTTP-2025-2011-0002044 & -000002----222-0011111
and in part by the National Research Foundation (NRF) of Korea grant funded by the Korea government (MSIT) (No. RS-2023-00212780)
REFERENCES
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, 1lya Mironov, Kunal Talwar, and Li Zhang Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference computer and communications security, pp. 308-318, 2016.
Panos Achlioptas, Olga Diamanti, Joannis Mitliagkas, and I Leonidas Guibas. Learning representations and generative models for 3d point clouds. In International conference on machine learning, pp. 40-49. PMLR, 2018.
Gunilla Borgefors. Distance transformations in arbitrary dimensions. Computer vision, graphics, and image processing, 27(3):321 345, 1984.
Lucas Caccia, Herke Van Hoof, Aaron Courville, and Joelle Pineau, Deep generative modeling lidar data. In 2019 JEEF/RSJ International Conference on Intelligent Robots and Systems (IROS), pp- 5034-5040. IEFE, 2019.
Angel X Chang. Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, al. Shapenet: An information-rich 3d model repository. arXiv preprint 1rtiv:1112.030111 2015.
Yuwei Cheng and Yimin Liu. Person reidentification based on automotive radar point clouds. IEEE Transactions on Geoxcience and Remote Sensing, 60:1-13, 2021.
Alexey Dosovitskiy and Thomas Brox. Inverting visual representations with convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016a.
Alexcy Dosovitskiy and Thomas Brox. Generating images with perceptual similarity metrics based on deep networks. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Infor- mation Processing Systems, volume 29. Curran Associates, Inc., 2016b. URL 7dc838 717893bcdeeeee 137 379933
beurips.cc/pppt /ttpripp..c///pper/////../../.
.dff
Mengnan Du, Ninghao Liu, Qingquan Song, and Xia Hu. Towards explanation of dnn- dasebased prediction with guided feature inversion. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1358-1367, 2018.
Mihai Dusmanu, Johannes L Schonberger, Sudipta N Sinha, and Marc Pollefeys. Privacy-preserving image features via adversarial affine subspace embeddings. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 14267-14277, 2021.
GDPR EU, General data protection regulation, 2016.
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE conference computer vision and pattern recognition, pp. 3354-3361. IEEE, 2012.
Cyril Goutte and Eric Gaussier. A probabilistic interpretation precision, recall and f-score, with implication for evaluation. In David E. Losada and Juan M. Fernández-Lumi (ods.). Advances in Information Retrieval, pp. 345-359 359, Berlin, Heidelberg, 2005. Springer Berlin Heidelberg. ISBN 978- 1
Benjamin Graham and Laurens Van der Maaten. Submanifold sparse convolutional networks. arXiv preprint arXiv: 070600.307. 2017.
Longhua Guo, Mianxiong Dong, Kaoru Ota, Qiang Li, Tianpeng Ye, Jun Wu, and Jianhua Li. secure mechanism for big data collection large scale internet vehicle. IEEE Internet of Things Journal, 4(2): 601-610, 2017.
Otkrist Gupta and Ramesh Raskar. Distributed learning of deep neural network over multiple agents. Journal of Network and Computer Applications, 116:1-8, 2018.
11

---

Published as a conference paper at ICLR 2025
Mengxi Hanyao, Yibo Jin, Zhuzhong Qian, Sheng Zhang, and Sanglu Lu. Edge-assisted online on-device object detection for real-time video analytics. IEEE INFOCOM 2021-IEEE Conference on Computer Communications, pp. 1-10. IEEE, 2021.
Zitian Huang, Yikuan Yu, Jiawen Xu, Feng Ni, and Xinyi L... Pf-net: Point fractal network for 3d point cloud completion. In Proceedings of the IEEF//VV conference on computer vision and pattern recognition, pp- 7662-7670, 2020.
D.P. Huttenlocher, G.A. Klanderman, W.J. Rucklidge. Comparing images using the hausdorff distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(9):850 863, 1993. doi: 10.1109/34.232773.
Sunwook Hwang. Youngseok Kim, Seongwon Kim, Saewoong Bahk, and Hyung-Sin Kim. Upcycling: Semi- supervised 3d object detection without sharing raw-level unlabeled scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 23351-2000 233611, October 2023.
Pileun Kim, Jingdao Chen, and Yong K Cho. Slam-driven rohotic mapping and registration 3d point clouds. Automation in Construction, 89:38-48, 2018.
Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv: (10..6690000722 02907, 2016.
Alex H Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang. and Oscar Beijbom. Pointpillars: Fast encoders for object detection from point clouds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12697-12705, 2019.
Xinke Li, Zhirui Chen, Yue Zhao, Zekun Tong, Yabang Zhao, Andrew Lim, and Joey Tianyi Zhou. Pointba: Towards backdoor attacks in 3d point cloud. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 16501, 2021.
Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar, Focal loss for dense object detection. In Proceedings of the IEEE International Conference Computer Vision (ICCV), Oct 2017.
Baoyuan Liu, Min Wang, Hassan Foraosh, Marshall Tappen, and Marianna Pensky. Sparse convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 806-814, 2015.
Sicong Liu, Junzhao Du, Anshumali Shrivastava, and Lin Zhong, Privacy adversarial network: representation learning for mobile data privacy. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3(4):1-18, 2019.
Yuhui Lu, Zhongxi Chen, and Mingbo Zhao. 3d objective detection for autonomous driving based on two-stage approach. In 2022 IEFE International Symposium Product Compliance Engineering-Asic (ISPCE-ASIA), pp. 1-5. IEEE, 2022.
Shitong Luo and Wei Hu. Diffusion probabilistic models for 3d point cloud generation. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2837-2845, 2021.
Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representations inverting them. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5188-5196, 2015.
Priyanka Mandikal and Venkatesh Babu Radhakrishnan. Dense 3d point cloud reconstruction using deep pyramid network. 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1052-1060. IEFE, 2019.
Priyanka Mandikal, KL Navaneet, Mayank Agarwal, and R Venkatesh Babu. 3d-Imnet: Latent embedding matching for accurate and diverse 3d point cloud reconstruction from a single image. arXiv preprint arXiv: T.X.vv/807.07796 2018.
Luke Kyriazi, Christian Rupprecht, and Andrea Vedaldi. Pc2: Projection-condtioone point cloud diffuusin for single-image 3d reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12923-12932. 2023.
Trix Mulder and Nynke Vellinga. Exploring data protection challenges of automated driving. Computer Law & Security Review, 40: 105530, 2021.
Tony Ng. Hyo Jin Kim, Vincent Lee, Daniel DeTone, Tsun- Yi Yang, Tianwei Shen, Eddy 1lg. Vassileios Balntas, Krystian Mikolajczyk, and Chris Sweeney. Ninjadese: content content-concealle concealing visual descriptors via adversarial learning. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 12797-12807, 2022.
12

---

Published as a conference paper at ICLR 2025
Keondo Park. You Rim Choi, Inhoe Lee, and Hyung-Sin Kim. Pointsplit: Towards on-device 3d object detection with heterogeneous low-power accelerators. In Proceedings of the 22nd International Conference on Information Processing in Sensor Networks, pp. 67-81, 2023.
Francesco Pittaluga. Sanjeev Koppal, Sing Bing Kang, and Sudipta N Sinha. Revealing scenes by inverting structure from motion reconstructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 145,154,22 2019.
Charles Ruizhongtai Qi. Li Yi. Hao Su, and Leonidas J Guibas. Pointuet++: Deep hierarchical feature learning on point sets in metric space. Advances neural information processing systems, 30, 2017.
Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. Pointrenn: 3d object proposal generation and detection from point cloud. In Proceedings the IEEE/CVF conference on computer vision and pottern recognition. pp. 770-779, 2019.
Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. Pv-reun: voint-voxel feature abstraction for 3d object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10529 10538, 2020.
Weijing Shi and Raj Rajkumar. Point-gmi gnn: Graph neural network for 3d object detection in point cloud. In Proceedings of the IEEE/CVF conference computer vision and pottern recognition, pp. 1711-1719, 2020.
Abhishek Singh, Praneeth Vepakomma, Otkrist Gupta, and Ramesh Raskar. Detailed comparison of communica- tion efficiency of split lcarning and fedcrated learning. arXiv preprint (9ti:0999009900 2019a.
Akash Deep Singh, Sandeep Singh Sandha, Luis Garcia, and Mani Srivastava. Radhar: Human activity recognition from point clouds generated through millimeter wave radar. In Proceedings of the 3rd ACM Workshop on Millimeter-wave Networks and Sensing Systems, pp. 51-56, 2019h.
Zhenbo Song, Wayne Chen, Dylan Campbell, and Hongdong Li. Deep novel view synthesis from colored 3d point clouds. In Computer Vision ECCV 2020: 16th European Conference, Glasgow, UK, August 23- 28, 2020, Proceedings, Part XXIV 16, pp. 1-17. Springer, 2020.
Bernd Carsten Stahl and David Wright. Ethics and privacy in and big data: Implementing responsible research and innovation. IEEE Security & Privacy, 16(3)26 33, 2018.
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et Scalability perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp- 2446-2454, 2020.
Pei Sun, Mingxing Tan, Weiyue Wang, Chenxi Liu, Fei Xia, Zhaoqi Leng, and Dragomir Anguelov. Swformer: Sparse window transformer for 3d object detection in point clouds. In European Conference on Computer Vision, pp. 426-442. Springer, 2022.
OpenPCDet Development Team. Openpedet: An open-source toolbox for 3d object detection from point clouds. https://github: com aapen--mllbbl Open PCDDt 2020.
Hugues Thomas, Charles R Qi. -ean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas. Kpconv: Flexible and deformable convolution for point clouds. In Proceedings of the IEFF/CVF international conference on computer vision, pp. 6411-6420, 2019.
Diego Valsesia, Giulia Fracastoro, and Enrico Magli. Learning localized generative models for 3d point clouds via graph convolution. In International conference on learning representations, 2018
Praneeth Vepakomma, Otkrist Gupta, Tristan Swedish, and Ramesh Raskar. Split lcarning for health: Distributed deep lcarning without sharing raw patient data. arXiv preprint 1812::8112000500 2018
Tsun-Hsuan Wang, Sivabalan Manivasagam, Ming Liang, Bin Yang. Wenyuan Zeng, and Raquel Urtasun. V2vnet: Vehicle-to-vehicle communication for joint perception and prediction. In Computer Vision-ECCC ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings. Part II 16, pp. 605-621. Springer, 2020.
Xianlong Wang, Minghui Li, Wei Liu, Hangtao Zhang, Shengshan Hu, Yechao Zhang, Ziqi Zhou, and Hai Jin. Unlcamable 3d point clouds: Class-wise transformation is all you need. In In Advances in Neural Information Processing Systems (NeurIPS), 2024, 2024a.
Xianlong Wang, Minghui Li, Peng Xu, Wei Liu, Leo Yu Zhang, Shengshan Hu, and Yanjun Zhang Pointapa: Towards availability poisoning attacks in 3d point clouds. In Eumpean Symposium on Research in Computer Security, pp. 125-145. Springer, 2024b.
13

---

Published as a conference paper at ICLR 2025
Xin Wen, Tianyang Li, Zhizhong Han, and Yu-Shen Liu. Point cloud completion by skip-attention network with hierarchical folding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1939-1944 1948, 2020.
Wenxuan Wu, Zhongang Qi, and Fuxin. Pointconv: Deep convolutional networks on 3d point clouds. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp. 9621 2019.
Zhu Xiao, Jinmei Shu, Hongbo Jiang, Geyong Min, Hongyang Chen, and Zhu Han. Perception task offloading with collaborative computation for autonomous driving. IEEE Journal on Selected Areas in Communications, 41(2):457 473, 2022.
Yuwen Xiong, Wei-Chiu Ma, Jingkang Wang, and Raquel Urtasun. Learning compact representations for lidar completion and generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patter Recognition, pp. 1074-1083 2023.
Runsheng Xu, Hao Xiang. Zhengzhong Tu, Xin Xia, Ming- Hsuan Yang, and Jiaqi Ma. V2x-vit: Vehicle-to- everything cooperative perception with vision transformer. In European conference on computer vision, pp- 107-124. Springer, 2022.
Hanyu Xue, Bo Liu, Ming Ding. Tianqing Zhu, Dayong Ye, Li Song, and Wanlei Zhou. Dp-image: Differential privacy for image data feature space, 2023.
Yan Yan, Yuxing Mao, and Bo Li. Second: Sparsely embedded convolutional detection. Sensors, 18(10):3337, 2018.
Guandao Yang, Xun Huang. Zekun Hao, Ming- Yu Liu, Serge Belongie, and Bharath Hariharan, Pointtlow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the IEFE/CVF international conference on computer vision, pp. 4541-4550 2019a.
Ziqi Yang, Jiyi Zhang, Ec-Chien Chang, and Zhenkai Liang. Neural network inversion adversarial setting via background knowledge alignment. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pp. 225-240, 2019b.
Tianwei Yin, Xingyi Zhou, and Philipp Krahenbuhl. Center-based 3d object detection and tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11784-11793, 2021.
Haibao Yu, Yizhen Luo, Mao Shu, Yiyi Huo, Zehang Yang, Yifeng Shi, Zhenglong Guo, Hanyu Li, Xing Hu, Jirui Yuan, et al. Dair-v2x: A large-scale dataset for vehitue-ifrrasruucure cooperative 3d object detection. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 21361-21370, 2022.
Xumin Yu, Yongming Rao, Ziyi Wang. Zuyan Liu, Jiwen Lu, and Jie Zhou. Pointr: Diverse point cloud completion with geometry-aware transformers. Proceedings of the IEEE/CVF international conference on computer vision, pp. 12498-122 12507, 2021.
Jiang Zhang, Lillian Clark, Mathew Clark, Konstantinos Psounis, and Peter Kairouz, Privacy-utility trades in crowdsourced signal map obfuscation. Computer Networks, 215:109187, 2022.
Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxian Wang. Bo Li, and Dawn Song. The secret revealer: Generative model-innersion attacks against deep neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 253-261, 2020.
Ziyu Zhang, Feipeng Da and Yi Yu. Data-free point cloud network for 3d face recognition. arXiv preprint arXiv:1911.044 047331, 2019.
Xuejun Zhao, Wencan Zhang, Xiaokui Xiao, and Brian Lim. Exploiting explanations for model inversion attacks. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 682-692, October 2021.
Yin Zhou and Oncel Tuzel. Voxelnet: End-to-end learning for point cloud based 3d object detection. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 4490 4499, 2018.
Yufan Zhou, Haiwei Dong, and Abdulmotaleb El Saddik. Learning to estimate 3d human pose from point cloud. IEEE Sensors Journal,
2020.
Vlas Zyrianov, Xiyue Zhu, and Shenlong Wang. Learning to generate realistic lidar point clouds. In European Conference on Computer Vision, pp. 17-35. Springer, 2022.
14


Reviews:
[{"review_id": "io2b5RH2dK", "reviewer": "Reviewer_1XWX", "summary": "ConcreTizer, a novel model inversion attack specifically designed for restoring 3D point cloud scenes from disrupted 3D feature data, is presented in this paper.  The unique challenges in 3D point cloud inversion, such as the inherent sparsity of the data and the ambiguity between empty and non-empty voxels after voxelization have been identified.  The authors address these challenges by incorporating two key techniques: Voxel Occupancy Classification (VOC) to distinguish between empty and non-empty voxels, and Dispersion-Controlled Supervision (DCS) to mitigate the dispersion of non-empty voxels across feature extractor layers. Evaluations on widely used 3D feature extractors and benchmark datasets demonstrate the effectiveness of ConcreTizer in restoring original 3D point cloud scenes.  This work highlighted the vulnerability of 3D data to inversion attacks.", "questions": "Can the authors comment on how to improve the exploration of the defense strategies beyond what is done here? What are other directions and what are inherent challenges?\n\nAny comments on whether the identified model inversion risks apply to point-based and graph-based feature extractors?\n\nAny comments on how the identified privacy risks would compromise privacy or security in more concrete scenarios?  Why would anyone care about this?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper presents the first in-depth study of model inversion attacks for 3D point cloud data.  The proposed ConcreTizer appears to be novel, designed to tackle the unique challenges associated with 3D point cloud inversion.    \n\nThe paper is well-written with a clear and concise explanation of the problem, the proposed solution, and the experimental results. The experiments are comprehensive and well-designed, demonstrating the effectiveness of ConcreTizer across various datasets and 3D feature extractors.    \n\nThe findings of this paper have significant implications for the privacy and security of 3D point cloud data. The authors demonstrate that 3D data is vulnerable to inversion attacks, even when shared in the form of intermediate features.  This highlights the urgent need for robust defense strategies to protect sensitive information in 3D data. The proposed ConcreTizer method serves as a valuable tool for assessing the effectiveness of such defense strategies.", "weaknesses": "While the paper emphasizes the need for defense strategies, it only briefly explores the use of Gaussian noise addition as a potential defense mechanism.  A more in-depth investigation of various defense strategies, including their effectiveness and limitations, would further strengthen the paper's contribution.  For example, I would recommend exploring adversarial training, differential privacy, or feature obfuscation techniques. I would recommend an analysis of the trade-offs between privacy protection and utility preservation in these defense mechanisms.\n\nConcreTizer method is specifically designed for voxel-based feature extractors.  While these extractors are commonly used in autonomous vehicles, exploring the applicability of the method to other types of 3D feature extractors, e.g., point-based and graph-based would help broaden its scope and impact.   I recommend discussing how the key components of ConcreTizer (VOC and DCS) might need to be adapted for point-based or graph-based extractors, and highlight challenges they anticipate in extending the method to these architectures.\n\nWhile the paper discusses the potential privacy risks associated with 3D point cloud data, it could benefit from a more detailed discussion of the real-world implications of these risks. This could include specific examples of how restored 3D scenes could be used to compromise privacy or security.  I also recommend the authors to provide concrete examples of privacy breaches that could occur with restored 3D scenes, such as identifying individuals, revealing sensitive infrastructure details, or inferring behavioral patterns. It's useful to discuss potential legal or ethical implications of these privacy risks in the context of autonomous vehicles and smart cities.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7INTChXKae", "reviewer": "Reviewer_3vLL", "summary": "This paper presents the first comprehensive study of model inversion attacks in 3D point cloud reconstruction tasks, introducing an attack method named ConcreTizer. ConcreTizer utilizes Voxel Occupancy Classification (VOC) to distinguish between zero-padded empty voxels and non-empty voxels containing valid points at the origin (0, 0, 0), along with Dispersion-Controlled Supervision (DCS) to enforce sparsity in the recovered 3D point cloud data. Extensive quantitative and qualitative experiments on open-source datasets, such as KITTI and Waymo, validate the effectiveness of this inversion attack and underscore the privacy vulnerabilities and the critical need for robust defenses in 3D point cloud data applications.", "questions": "There are ambiguities in the presentation. For instance, in Figure 2, the white areas, as suggested by the context, represent actual point cloud data, which decreases with increasing layer depth. However, the caption of Figure 2 states, “As the layer depth increases, the number of restored points increases rapidly, ...” which contradicts the visual representation and may confuse readers trying to align the illustration with the description.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- ConcreTizer represents the first exploration of inversion attacks on 3D point cloud data, providing valuable insights and inspiration for further research.\n- The paper demonstrates ConcreTizer’s effectiveness across various datasets and feature extractors, achieving impressive point cloud reconstruction results. The breadth and diversity of experiments enhance the credibility of the findings.", "weaknesses": "1. The attack scenario in this paper lacks practical significance. One of the core innovations proposed by the paper is the introduction of VOC to differentiate between empty voxels and those located at the (0,0,0), as they share the same representation. However, a simpler solution exists to address this issue, such as adding an \"occupied\" label when voxelizing the point cloud data, which would render the VOC setup unnecessary.\n2. The paper contains some ambiguities in its expression, making it difficult to understand. See Question 1 for further details.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ffNf76bjUm", "reviewer": "Reviewer_VptB", "summary": "The paper proposed the first study of model inversion attacks for 3D point cloud data. The primary contributions of our work include (1) identifying the challenges associated with the interaction between 3D point cloud characteristics and voxel-based feature extractors, (2) introducting ConcreTizer that utilizes Voxel Occupancy Classification (VOC) and Dispersion-Controlled Supervision (DCS) to restore original 3D point clouds from their voxel-based features. (3) Experiments demonstrate the superior performance of ConcreTizer compared to a traditional method and GAN-based method. The findings highlight the vulnerability of 3D point clouds to inversion attacks and underscore the need for enhanced defensive strategies in privacy-sensitive applications.", "questions": "Please refer to the weakness part.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.\tOriginality: The paper introduces ConcreTizer, the first approach to 3D point cloud restoration from voxel-based features, highlighting previously unexamined vulnerabilities. \n2.\tClarity: The paper is well-organized.\n3.\tSignificance: By demonstrating the risks of model inversion attacks on voxel-based 3D features, this work has meaningful implications for privacy in applications like autonomous driving, emphasizing the need for robust defenses in this domain.", "weaknesses": "1.\tFocus on Voxel-based Features Only:\nThe paper’s focus is solely on voxel-based feature inversion, with no consideration for point-based or projection-based features. Broadening to other feature types or discussing ConcreTizer’s potential for generalization would enhance applicability.\n2.\tLimited Comparison Methods:\nConcreTizer is mainly compared with Point Regression and UltraLiDAR. Including more inversion methods, such as recent GAN-based or diffusion-based approaches, would provide a more thorough evaluation and highlight ConcreTizer's strengths more effectively.\n3.\tLimited Validation Scope:\na)\t3D Object Detection Evaluation: Only PointPillar is used to assess detection on restored scenes. Testing with other models (e.g., CenterPoint, PV-RCNN…) would provide better insight into generalizability.\nb)\tEffect of DCS Instances in Ablation Study: The chosen values for DCS instances (1, 2, 4, 10) are limited and unevenly spaced. Testing a broader, more evenly distributed range could offer clearer insights into optimal DCS configuration.\n4.\tWriting:\na)\tWhy the results for VoxelResBackbone with the Waymo dataset are not included?\nb)\tIt’s unclear if \"VOC alone\" in the component-wise analysis ablation study refers solely to using the regression loss function. Explicit clarification would help. \nc)\tThe phrase, “Since the restoration task does not require separate labeling, it is possible to train using open source datasets or self-collected data,” could be streamlined as the first part might appears redundant.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GmHndeKlSd", "reviewer": "Reviewer_8SUr", "summary": "The paper highlights the growing privacy concerns surrounding the use of 3D point cloud data in autonomous vehicles due to the sensitive information it contains. While model inversion attacks have been well-studied in 2D data contexts, their application to 3D point clouds has largely been unexplored. To address this, the authors present an in-depth study of model inversion attacks aimed at reconstructing 3D point cloud scenes. They identify challenges such as the inherent sparsity of 3D data and the ambiguity between empty and non-empty voxels after voxelization, which is worsened by voxel dispersion across feature extractor layers. To tackle these issues, they introduce ConcreTizer, a model inversion attack specifically designed for 3D point cloud data, incorporating Voxel Occupancy Classification and Dispersion-Controlled Supervision. Experiments on popular 3D feature extractors and datasets like KITTI and Waymo demonstrate ConcreTizer's effectiveness in restoring disrupted 3D point cloud scenes, underscoring the vulnerability of 3D data to such attacks and the urgent need for robust defense strategies.", "questions": "As a minor suggestion, the authors could revise the statement in Line 47 to make this claim “However, efforts to protect raw data have primarily focused on 2D domains, rather than 3D data.“ more accurate. Since recently there is already some parallel studies that mainly focus on protecting 3D raw data privacy against unauthorized training [5], the authors can revise this sentence to suggest the difference between existing 3D works and this paper's concentration view and the importance of studying model inversion attacks in 3D point clouds.\n\n[5] Unlearnable 3D Point Clouds: Class-wise Transformation Is All You Need. NeurIPS 2024", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.Good writing. This paper is well-written and easy to understand for even outliers like me, since I am not an expert in the research domain of model inversion attacks.\n\n2.Nice figure presentations. The nice figures including Figures 1-3 make the paper more clearly and visually pleasing.\n\n3.Well-motivated scheme. The authors have justified the motivation and key intuition behind proposed research problem and solution.", "weaknesses": "1.More analysis about the failures of existing inversion attacks are needed. In the Introduction section, Lines 71-72, the authors just demonstrated that conventional attacks failed, however, the authors did not provide more insights behind this phenomenon. A detailed explanation will help us get a better understanding of the reason why the proposed attack is effective.\n\n2.Limited scope of proposed attacks. The authors said that, \"we delve into the phenomena that arise when the voxel-based feature extractors handle 3D point cloud scenes\", however, not all point cloud networks need to realize voxelization during feature extracting [1]. Therefore, it seems that the first challenge cannot represent the majority point cloud inversion attack scenes, limiting the effectiveness scope of the proposed attack scheme.\n\n3.I encourage the authors to conduct more experiments to make results more convincing, since the page limit is 10 in ICLR 2025, providing enough space to demonstrate more results. For instance, the authors are supposed to employ more 3D point cloud datasets (which may not only focus autonomous driving datasets)\n\n4.More defense baselines are supposed to be used for evaluation. Many recent works about point cloud adversarial attacks [2-4] have  employed  popular defense schemes like random 3D data augmentations, SOR, and SRS to evaluate the point cloud attack scheme's robustness. Thus, I will appreciate it if the authors can provide more results about defense evaluations.\n\n[1] Dynamic graph cnn for learning on point clouds. TOG 2019\n\n[2] PointAPA: Towards availability poisoning attacks in 3D point clouds. ESORICS 2024\n\n[3] PointCRT: Detecting backdoor in 3D point cloud via corruption robustness. ACM-MM 2023\n\n[4] PointBA: Towards backdoor attacks in 3D point cloud. ICCV 2021", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)