Geometry Cloak: Preventing TGS-based 3D Reconstruction from Copyrighted Images

Authors: qi  song
ziyuan  luo
ka chun cheung
simon  see
renjie  wan
Venue: NeurIPS_cc_2024_Conference
Published: 2024-04-30 09:20:00

Abstract: Single-view 3D reconstruction methods like Triplane Gaussian Splatting (TGS) have enabled high-quality 3D model generation from just a single image input within seconds. However, this capability raises concerns about potential misuse, where malicious users could exploit TGS to create unauthorized 3D models from copyrighted images. To prevent such infringement, we propose a novel image protection approach that embeds invisible geometry perturbations, termed ``geometry cloaks'', into images before supplying them to TGS. These carefully crafted perturbations encode a customized message that is revealed when TGS attempts 3D reconstructions of the cloaked image. Unlike conventional adversarial attacks that simply degrade output quality, our method forces TGS to fail the 3D reconstruction in a specific way - by generating an identifiable customized pattern that acts as a watermark. This watermark allows copyright holders to assert ownership over any attempted 3D reconstructions made from their protected images. Extensive experiments have verified the effectiveness of our geometry cloak.

Main paper: 
 introduction: 
 With the increasing importance of 3D assets, several methods have been proposed to reconstruct or generate 3D models from single 2D images. Combining with Tensorial Radiance Fields [2] , Triplane-based Gaussian Splatting (TGS) [52] presents a compelling approach for producing 3D models from single-view images. However, malicious users could potentially exploit TGS [52] to generate 3D models from single-view images without authorization, posing a threat to the interests of image copyright owners. To address this issue, it is essential for image owners to implement measures that can safeguard their copyrighted images from being used by TGS [52] .Digital watermarking [6, 51] is an effective way of claiming the copyright of digital assets. Thus, one potential method for safeguarding copyrighted images is to embed unique messages within images intended for building a 3D model and then extract the embedded messages from the reconstructed 3D model. However, previous methods have proven it is difficult to transfer the embedded copyright messages in 2D images into 3D models [31, 19] . Moreover, even if we can embed and extract copyright messages, the 3D models might have already been used by others before the copyright being claimed.To prevent unauthorized 3D reconstruction from copyrighted images via TGS [52] , a possible approach is to incorporate adversarial perturbations [11, 28] into input images intended for TGS. Images can be easily reconstructed into 3D models by malicious users with TGS [52] , posing a threat to the copyright of the image owner. (b) Digital Watermarking offers a solution by embedding copyright messages into the view-image before 3D reconstruction. However, the embedded message cannot be extracted from novel rendered views. (c) Geometry Cloak. Our geometry cloak utilizes the disturbance-prone components of TGS, achieving view-specific watermark embedding. Our method can compromise the unauthorized reconstructed 3D model while providing a verifiable pattern for copyright claim.Adversarial methods [11, 28] have already achieved promising results by introducing disturbances into input images to prevent neural models from functioning correctly. When it comes to TGS [52] , a straightforward solution is to incorporate such adversarial perturbations into input images by maximizing the difference between rendered and ground truth views like previous methods [9, 13] . However, those methods [9, 13] intuitively focus on disturbing rendered results, ignoring perturbationprone components of TGS [52] . As a result, those conventional adversarial perturbations can only lead to limited changes to the reconstructed results [9, 13] , which may still be used for several illicit applications. Besides, simple adversarial perturbations can only disrupt the results of 3D reconstruction in an uncontrollable manner but do not support traceability. The users have difficulties claiming copyright after the disturbance-affected 3D generation of images.We envision a novel scenario where copyrighted images can induce TGS to generate compromised content with an identifiable pattern. To achieve this goal, a naive solution is to employ image cloaking. Traditionally, image cloaking techniques [23, 37, 41, 35] are used to prevent the image from malicious editing by diffusion models [33] . The cloaking integrated into images can shift image features into another domain through adversarial perturbations. Consequently, this manipulation directs diffusion models to generate predetermined specific outcomes. If malicious users attempt to create 3D models using these protected images, the resulting compromised 3D models will be unusable. Besides, the identifiable pattern exhibited can help the image owner assert their copyrights in the event of legal inquiries.However, unlike image cloak methods [35, 23] against diffusion models [33] , simply perturbing image features cannot effectively induce TGS (as shown in Fig. 1 ). Image features in TGS show strong robustness against disturbances, even under relatively strong attack settings. Thus, the key becomes how to identify perturbation-prone components in TGS [52] and capitalize on this weakness to induce reconstructed results. TGS [52] contains image and geometry features during its 3D model reconstructions. Previous works [8, 3] have shown the geometry feature in 3D Gaussian Splatting [18] is easier to be manipulated with external operations. Therefore, we wonder if it is possible to manipulate the estimated point cloud process with an invisible disturbance. Based on this simple observation, we propose embedding invisible adversarial perturbations as a geometry cloak on images intended for TGS [52] , which can affect and manipulate estimated point clouds in the process of TGS's 3D reconstruction.As shown in Fig. 2 , we introduce a geometry cloak, which is carefully crafted to induce TGS [52] to fail 3D model generation and reveal our embedded pattern. To induce the TGS to reveal the embedded patterns, we propose a view-specific Projected Gradient Descent [28] (view-specific PGD) strategy by optimizing the distance between the projected point cloud view and predefined patterns. The PGD iteratively updates the geometry cloak by minimizing CD loss, ultimately revealing the desired view and uncovering the hidden patterns within the TGS [52] . When malicious users reconstruct 3D models using the protected images with TGS, the compromised geometry information causes TGS to reveal the embedded message. Unlike traditional copyright protection methods like digital watermarking [25, 51, 39] , which typically entail additional procedures for extracting the watermark post-use. Our approach directly transforms TGS [52] into a message disclosure tool by inducing it to yield a specific stylistic outcome, facilitating the verification of image ownership rights. In summary, our main contributions are threefold:• We propose the concept of "geometry cloaking", which can prevent unauthorized image-to-3D generation by TGS [52] , and it also leaves a verifiable copyright pattern.• Our geometry cloaking technique explores the perturbation-prone components of Triplanebased Gaussian Splatting and utilizes this vulnerability to achieve the protection of images.• We propose a view-specific PGD strategy, which can embed identifiable patterns into a specific view of the reconstructed 3D model.As our approach attacks the geometry features that are widely used in various single-image-to-3D approaches, it demonstrates generalization capability to other GS-based single-view to 3D approaches like LGM [40] . The results can be found in our experiments.2 Related work 
building 3d models from single images: 
 There has been a surge of research on learning to generate novel views from a single image [47, 36, 52] . This task aims to infer the 3D structure of a scene from a single 2D image and render new perspectives, enabling applications in virtual reality, augmented reality, and computer-aided design. One approach is NeuralLift-360 [47] , which incorporates a CLIP loss [32] to enforce similarity between the rendered image and the input image. Another method, 3DFuse [36] , fine-tunes the Stable Diffusion model [33] with LoRA [14] layers and a sparse depth injector. Recent work like Zero123 [24] takes a different approach by fine-tuning the latent stable diffusion model [33] to generate novel views based on relative camera pose. Recently, Triplane-based neural rendering methods [2, 52, 12] adopt a novel approach to model and reconstruct radiance fields. Unlike NeRF, which uses pure MLPs, Tensorial Radiance Fields (TensoRF) [2] consider the full volume field as a 4D tensor and propose to factorize the tensor into multiple compact low-rank tensor components for efficient scene modeling. Combing TensoRF [2] with novel 3D Gaussian Splatting [18] , Zou et al.propose a Triplane-base Gaussian Splatting [52] , which can obtain a 3D model from single-view image within seconds [52] . With continued research and development, more impressive and realistic 3D reconstructions in the future will enable immersive experiences and streamlined design workflows. The advancements in creating 3D models from a single-view image offer significant potential for diverse applications and digital assets. Consequently, it is essential to address the protection of copyrighted images to prevent their misuse in generating 3D models.Adversarial attacks [11, 28] have become a significant concern in the field of computer vision [22, 16] . These attacks aim to deceive machine learning models by introducing adversarial perturbations to the input data, leading to incorrect predictions such as misclassification [11] . While initially studied in the context of image classification models, adversarial attacks have also been explored in other domains [35, 23] . For Neural Radiance Fields (NeRFs) [29] , several works have proposed methods to perturb or enhance the original NeRF framework. NeRFs are a class of models that can synthesize high-quality 3D scenes from 2D images by learning the scene's volumetric representation and appearance. However, NeRFs are also vulnerable to adversarial attacks. Several recent works have investigated different techniques to perturb or enhance the original NeRF framework using adversarial attacks. NeRFool [9] presents an approach to manipulate NeRFs by perturbing the scene's geometry and appearance using adversarial attacks. NeRFTargeted [13] focuses on targeted perturbations in NeRFs, allowing for the optimization of scene parameters to generate desired target images. ShieldingNeRF [46] introduces a technique to protect sensitive information in NeRF-generated views by introducing obfuscating perturbations. These works demonstrate the potential of perturbing NeRFs for various objectives, including adversarial attacks, targeted image manipulation, and privacy protection, contributing to the advancement of NeRF-based models [2, 29, 27] in computer graphics and computer vision tasks. However, with the promising developments in 3DGS [18] , research on defending against attacks on Gaussian Splatting renderings remains an area that requires further investigation.With the rapid advancement of AI models, the risk of misuse has raised concerns, particularly in the malicious use of public data [43, 42, 50, 17] . Researchers advocate a proactive approach to prevent such misuse by adding subtle noise to images prior to publication. This technique aims to disrupt attempts at exploitation [21, 26, 20] . An important application of image cloaking is to prevent privacy violations from face recognition systems [38, 4, 15] . By introducing noise patterns to facial regions, these methods degrade face recognition model performance while maintaining visual quality. Image cloaking can also thwart image manipulation through GAN-based techniques like DeepFakes [30] by corrupting latent representations. Recent image cloaking methods [37, 23, 41] focus on protecting the copyrighted image from misuse by stable diffusion models. These methods disrupt artistic mimicry and harmful personalized images, aiming to prevent unauthorized exploitation. For example, GLAZE [37] and AdvDM [23] primarily focus on disrupting artistic mimicry and harmful personalized images generated by text-to-image models [10] . Anti-Dreambooth [41] concentrates on fine-tuning DreamBooth [34] for malicious face editing. These techniques aim to cloak input images in a way that disrupts the model's ability to generate personalized content while preserving the overall visual quality. Existing methods focus on protecting the copyrighted image from misuse by disturbing image features. By contrast, our method focuses on preventing copyrighted from being 3D reconstructed by TGS [52] without authorization, facing 3D scenes and complex copyright validity verification settings.3 Preliminaries of TGS TGS [52] introduces a novel hybrid 3D representation that integrates an explicit point cloud with an implicit triplane, enabling efficient and high-quality 3D object reconstruction from single-view images. As shown in Fig. 2 , the representation consists of a point cloud P ∈ R N ×3 providing explicit geometry, and a triplane T ∈ R 3×C×H×W encoding an implicit feature field, where T = (T xy , T xz , T yz ) comprises three orthogonal feature planes. For a given position x ∈ R 3 from the point cloud, the corresponding triplane feature f t is obtained by trilinear interpolation and concatenation of features from the three planes:f t = interp(T xy , P xy ) ⊕ interp(T xz , P xz ) ⊕ interp(T yz , P yz ).(1) Utilizing this hybrid representation, the 3D Gaussian attributes like opacity α, anisotropic covariance (scale s and rotation q), and spherical harmonics coefficients sh are decoded from f t augmented with projected local image features f l using an MLP ϕ g :(∆x ′ , α, s, q, sh) = ϕ g (x, f t ⊕ f l ).(2) Together, these parameters parameterize the 3D Gaussian kernel attributes around the point x, enabling differentiable Gaussian splatting for rendering.TGS represents a cutting-edge approach to 3D object reconstruction from single-view images, combining explicit and implicit representations to achieve accurate and detailed reconstructions. By leveraging advanced decoding mechanisms and efficient rendering techniques, the model demonstrates superior performance in generating realistic 3D models with intricate geometry and textural details. 
methodology: 
 Overview. With the increasing capabilities of 3D reconstruction techniques like TGS [52] , there is a risk of malicious users exploiting these methods to generate 3D models from copyrighted images In order to make the reconstruction result show some distinguishable characteristics, we use projected gradient descent (PGD) [28] to iteratively optimize the reconstructed point cloud so that it has consistent characteristics with the target point cloud (Section 4.2).without authorization, infringing on the rights of image owners. We propose geometry cloak, a novel solution by embedding invisible perturbations on the input images intended for TGS [52] . These perturbations are crafted to induce TGS to fail the reconstruction in a distinct way, producing an identifiable pattern in the corrupted 3D output. This section presents the methodology of inhibiting 3D reconstruction of TGS [52] via the introduced geometry cloak. Our method consists of two stages: (1) Building verifiable geometry patterns (Section 4.1), and (2) Optimizing geometry cloak with view-specific PGD (Section 4.2).As illustrated in Fig. 2 , to obtain a 3D model from single-view images, TGS [52] encodes the single-view image I and its associated camera parameters into image features. Following this, a point cloud decoder is adopted to project image features onto the point cloud. Then, a triplane decoder converts the image feature into the triplane latent T . Finally, 3D Gaussians are decoded from triplane feature f t for novel view synthesis.Our methodology diverges by targeting the explicit geometry features of the point cloud. Perturbing the point cloud directly is inherently more effective due to the inherent vulnerabilities in the TGS reconstruction process. In TGS [52] , the point cloud offers a distilled and direct representation [18] of the scene's structure, making modifications on them quite evident [8, 3] . Besides, the point cloud is not only foundational geometry information but also typically helps subsequent processing to obtain the final output. Point clouds provide essential geometry information that is instrumental in the sampling of features from the latent triplane representation (Eq. ( 1)). Moreover, by introducing strategic alterations to the geometry feature, we bypass some of the inherent robustness found in image-level features (Table 1 ) and avoid the complex transformations between image and geometry spaces. This direct manipulation allows for more precise control over the adversarial impact, exploiting specific vulnerabilities in TGS and leading to more pronounced disruptions in its output.Different types of target geometry pattern. Based on the inherent vulnerabilities of point clouds, we propose two types of verifiable pattern (Fig. 3 ), including (1) Pre-defined patterns and (2) Customized patterns. Through these patterns, the reconstructed 3D model undergoes consistent changes in geometry and visual appearance (novel views).Pre-defined patterns are 2D point clouds that are transformed from alphanumeric characters images, creating a direct and straightforward representation of watermarks. To obtain these 2D point clouds, we segment the image of alphanumeric characters and record the coordinates of these alphanumeric 2) Customized patterns: In E 1 , we first extract the point cloud of the image that needs to be protected. In E 2 , we edit the acquired point cloud through text-guided methods like instructP2P [48] or open-source software meshlab [5] . characters' segmentation. Then, we sample the point from segmentation coordinates and form a 2D point cloud as pre-defined patterns.Customized patterns are a more personalized approach where users can selectively protect a certain part of an image. In this setting, we first extract a point cloud from the image that requires safeguarding. We adopt the same methods from TGS [52] to obtain the point cloud P in E 1 . In E 2 , users can refine this point cloud to select the area they wish to protect. Users can employ text-guided editing techniques like instructP2P [48] or turn to open-source software like MeshLab [5] to further shape and customize the point cloud. This process not only enhances the visual appeal of the point cloud but also embeds a layer of security by aligning it with specific textual instructions or user intents.In this section, by exploiting the vulnerability of TGS [52] , we present view-specific PGD to determine geometry cloak δ, which can minimize the distance between reconstructed point clouds in TGS and target pre-defined patterns.Geometry Cloak δ. Geometry cloak δ is crafted to mislead the TGS into a controlled 3D reconstruction with imperceptible perturbations into single-view images. It minimizes the difference to the target geometry pattern while preserving image fidelity perceptible to the human eye through adversarial training [28] . where E denotes the network that maps image into point cloud in TGS [52] , and ϵ represents the perturbation budget.View-specific PGD. To embed pre-defined patterns in a specific viewing direction, we develop a viewspecific PGD. The optimization iteratively adjusts the geometry cloak δ to make the projected point cloud closer to the pre-defined patterns while keeping the image visually similar to the source. The updates use gradient decent [28] on L CD with learning rate α, moving the cloaked image towards the target. This iterative process manipulates the TGS's perception while maintaining visual similarity as below:EQUATIONwhere Pi θ = Proj θ (E(I (i) + δ (i) ), denoting the projected point cloud at viewing direction θ from cloaked image (I (i) + δ (i) ), and P w stands a 2D point cloud watermarks. We encapsulate the geometry cloaking process in Algorithm 1. Our method uses the PyTorch framework on a single NVIDIA V100 GPU. Our geometry cloak is obtained through optimization (Section 4.2) using projected gradient descent [11] . We adopt a mask version of PGD [28] for calculating geometry cloak, as only the object in the image is used for 3D reconstruction without its background. The input image I (0) = I is initialized, and iteratively updated for N = 100 steps with a step size of α = 0.001. The loss is defined as the Chamfer Distance [1] between the predicted point cloud P and the target point cloud Ptar . For pre-defined patterns, we adopt the proposed view-specific PGD as we want to embed the 2D pre-defined patterns into a certain viewing direction θ. For customized patterns, we directly calculate the distance between customized patterns and predicted point clouds, as both are 3D point clouds. The updated image I (i) is clipped to the valid range [0, 1], and after N = 100 iterations, the final image Î = I (N ) are obtained as the geometry cloaked output. 
settings: 
 Dataset. To evaluate the performance of our method, we conduct experiments on the Google Scanned Objects (GSO) [7] and OmniObject3D (Omni3D) [45] datasets. Both datasets embody a large diversity of view images and provide a rich and varied set of data for assessing the performance of our method. For GSO [7] , we select 1 image view for each of the 1030 objects, resulting in 1030 images.For Omni3D [45] , we choose 5 image views from each of the 190 classes in Omni3D, resulting in a total of 950 images.Baselines. To investigate the effectiveness of our approach, we evaluate the impact of different perturbations on the reconstructed results by TGS [52] . We experiment with four types of perturbations strategy: (1) Gauss. noise, i.e., random Gaussian noise is added to the protected image. (2) Adv. image, i.e., adversarial attacks on image feature [9] ; (3) Geometry cloak w/o target, i.e., adversarial attacks on the point cloud. For Adv. image and geometry cloak w/o target, we directly use the norm value of features [35] as the optimizing loss. (4) Geometry cloak. For the geometry cloak, we randomly select letters "A-Z" and numbers "1-9" as the pre-defined pattern.Evaluation methodology. We report the quantitative metric between no perturbation and perturbed reconstructed results. Specifically, for visual quality, we report image similarity metrics: PSNR, SSIM [44] , LPIPS [49] . For geometry quality, we report the Chamfer Distance (CD) [1] . We further present the qualitative customized 3D reconstruction by inducing the point from the single-view image into another domain. To showcase the effectiveness of our geometry cloak, we have conducted experiments using various perturbation strategies to assess their capability in preventing 3D reconstruction. We report the visual (PSNR/SSIM/LPIPS) and geometry (CD distance) differences between 3D models reconstructed from perturbed images and unprotected images.Perturbing image/geometry features. To identify the perturbation-prone components of TGS, we compare the perturbation of implicit image features (Adv. image) and explicit geometry features (Ours w/o target) at different intensities norms [35] . As shown in Table 1 , the experimental results show that by attacking the image features, the reconstructed visual and geometry quality is only slightly compromised, with the reconstruction results hardly impacted. However, attacking the point cloud resulted in significant changes between the perturbed and the non-perturbed 3D model. We further provide qualitative results in Fig. 5 . The image features exhibit strong robustness to perturbations. Even with obvious adversarial perturbations on the image, it is difficult to affect the Pre-defined pattern. Our method is proposed to perturb explicit geometry and develop a viewspecific pre-defined pattern. To evaluate the impacts of our geometry cloak on reconstructing 3D models via TGS [52] , we randomly select letters "A-Z" and numbers "1-9" as the pre-defined patterns.As shown in Table 1 , we can see that after specifying the target point cloud, the reconstructed 3D model is comprised both in vision and geometry results, making the reconstructed model unusable.We further demonstrate the qualitative results in Fig. 6 , showing that the watermark message can be re-emerged in the specific view perspective. This indicates that our method can preserve identifiable embedded information while ensuring the integrity of the 3D model.Customized pattern. We also demonstrate the performance of our method when using customized patterns. In this setting, the users can selectively protect specific parts of the image that need not be reconstructed for customized protection. As shown in Fig. 6 , users can choose to remove certain object parts (e.g., tail, body, head). The results indicate that users can effectively influence the reconstruction results with our geometry cloak, further demonstrating that point clouds in TGS are susceptible to perturbation. We provide more results to show that our geometry cloak can be generalized to other GS-based single-view to 3D method [40] in the appendix.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: The main codes are provided in supplemental material. Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
conclusion: 
 We present a novel geometry cloaking approach to protect image copyrights from unauthorized 3D reconstruction with Triplane Gaussian Splatting (TGS). By embedding carefully optimized perturbations in the geometry feature space that encodes a customized watermark message, our method forces TGS to fail reconstruction in a distinct way -generating the watermarked pattern.Extensive experiments validate our strategy of focusing perturbations on the geometry components of TGS, which can reliably induce watermarks with invisible perturbations. Our geometry cloaking introduces a novel method for protecting copyrights tailored to the representations of single-view to 3D models.Limitations and broader impacts. The multifaceted nature of copyright protection requires that our method be developed and used responsibly, respecting the delicate balance between innovation and intellectual property rights. Collaboration across technology, legal, and policy sectors is essential to address these complexities. • The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully. (c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: The paper specifies all the training and test details.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [NA] Justification: The experiment metrics are calculated on large-scale images and point clouds.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: The paper provides sufficient information on the computer resources.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. Justification: The paper discusses potential positive societal impacts and negative societal impacts.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] [TODO] Justification: Not applicable as our work is not relevant. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: Proper credit is given to the creators or original owners of assets used in the paper, and the license and terms of use are explicitly mentioned and respected. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [NA] Justification: The answer NA means that the paper does not release new assets. Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix:  


Decision: The paper introduces Geometry Cloak, a method to protect copyrighted images from unauthorized 3D reconstruction using Triplane Gaussian Splatting (TGS). Reviewer kd52 appreciates the novel perturbation technique that degrades 3D geometry without affecting 2D image quality but notes its limited applicability beyond TGS. Reviewer ykxU commends the clear presentation and effectiveness, suggesting extensions to other models like LRM and LGM, and testing robustness under noise and compression. Reviewer inRn acknowledges the novelty but questions adaptability to other methods and potential impacts on legitimate image use. Reviewer rPf4 recognized the significance and thorough experiments but recommends adding quantitative metrics like PSNR. Reviewer u5Eb criticizes the limited scope, urging tests on additional reconstruction methods for broader practical effects.

The rebuttals clarifies the geometry cloak method extends beyond TGS to other 3D reconstruction methods like LGM, as these rely on explicit geometric features. They provide additional experiments demonstrating the method's effectiveness across various settings, addressing concerns about generalizability, robustness, computational resources, and impact on image quality. Overall the reviewers welcomed the changes except u5Eb, who stressed that the paper should be evaluated on the initial submission, not the rebuttal. However, other reviewers (and AC) all feel that the changes in the rebuttal has indeed broadened the scope of the paper and made the point valid.

Reviews: review: paper_summary: paper_summary: This paper presents a method for copyrights in 3D reconstruction, specifically targeting novel-view synthesis, rather than traditional 2D images. Recently, advancements in 3D reconstruction have been driven by neural radiance fields (NeRFs) and 3D Gaussian splatting (3D GS), both of which maintain 3D consistency. These methods primarily focus on learning RGB values from multiple posed images. Notably, recent developments have shown that single-view 3D reconstruction can be achieved with the support of pre-trained diffusion-based generative models.

Unlike 2D images, privacy preservation in 3D reconstruction has received less attention due to its recent development. This paper introduces a geometry cloak that perturbs 3D point cloud representations instead of 2D images. This perturbation prevents 3D reconstruction from 2D posed images without degrading the visual quality of the 2D images.

The technique is particularly supportive of 3D GS, enabling real-time novel-view synthesis, a capability not supported by existing NeRFs. To the best of our knowledge, this paper is the first to propose an adversarial attack to preserve copyright in 3D GS from a 3D reconstruction perspective.
strength_weakness: strength_weakness: 1. This paper heavily relies on the prior work of Tri-Plane Gaussian Splatting (TGS), which explicitly represents 3D point clouds to enhance geometric properties and employs tri-plane representation to encode 3D Gaussians. Without leveraging TGS, this study would not effectively address copyright issues in 3D reconstruction. This dependency indicates that the proposed approach is not applicable to a wide range of novel-view synthesis techniques, highlighting a limitation in its generalizability.

2.  While this paper demonstrates that the proposed method is insensitive to the degree of perturbation, it does not adequately explain this phenomenon in the context of privacy preservation. In the past, understanding the tendency of performance degradation is crucial and be well-explained through probabilistic analysis for the extent of perturbation. However, the experimental results presented in this paper do not support this concept.
questions: questions: 1. Could you show the slope of performance deviation when $\epsilon$ is less than 2? When $\epsilon = \{0, 1\}$, does the proposed method also exhibit a negative slope of performance in terms of PSNR, SSIM, and LPIPS?

2. Could you show the performance degradation of random noise when $\epsilon$ increases beyond 8? It should demonstrate that the proposed approach is more beneficial under strong perturbation. While $\epsilon=8$ indicates color disturbance, the context and shape in the 2D images do not change.

3. Could you present the ablation study on the influences of PGD?
limitations: limitations: Please refer to weakness.
paper_summary: paper_summary: The paper introduces a novel image protection approach called "Geometry Cloak" to prevent unauthorized 3D model generation from copyrighted images using single-view 3D reconstruction methods like Triplane Gaussian Splatting (TGS). The Geometry Cloak embeds invisible geometry perturbations into images, which are revealed as a customized message when TGS attempts 3D reconstructions, thus acting as a watermark for copyright assertion.
strength_weakness: strength_weakness: 1. Can the proposed method be extended to other image to 3D models, such as LRM [1], LGM [2]? The author could introduce the advantages of using TGS instead of other image to 3D models in terms of generation speed and quality, so that the readers can better understand its task scenario. 

[1] Lrm: Large reconstruction model for single image to 3d. In ICLR 2024.

[2] Large multi-view gaussian model. In ECCV 2024.

2. The robustness of the proposed method should be verified. For example, when Gaussian noise and JPEG compression are added to the protected image, can it still resist illegal theft?
questions: questions: Please refer to the weakness.
limitations: limitations: The authors have clearly presented their limitations.
paper_summary: paper_summary: The paper introduces a novel approach to protect copyrighted images from unauthorized 3D reconstructions using Triplane Gaussian Splatting (TGS). The method involves embedding invisible geometry perturbations, termed "geometry cloaks," into images. These cloaks cause TGS to fail in a specific way, generating a recognizable watermark, thus protecting the original content.
strength_weakness: strength_weakness: 1. The approach is tailored specifically to TGS-based 3D reconstruction. Can the geometry cloak technique be adapted or extended to protect against other 3D reconstruction methods beyond TGS?
2. What are the potential impacts on image quality for legitimate uses when these cloaks are applied?
3. The optimization process for generating geometry cloaks might introduce significant computational overhead, which is not thoroughly discussed in the paper.
questions: questions: See Weaknesses
limitations: limitations: Mentioned in the manuscript.
paper_summary: paper_summary: This paper proposes a novel method to protect copyrighted images from unauthorized 3D reconstruction using Triplane Gaussian Splatting (TGS). This topic is very interesting and highly valuable for preventing the misuse of copyrighted images. Their proposed method achieves protection by incorporating invisible geometry perturbations into the image, which is easy to deploy with less cost. The extensive experimental results have demonstrated the effectiveness of their approach in preventing unauthorized 3D reconstructions while embedding a verifiable pattern. This paper can provide a contribution to addressing the growing issue of image abuse and raise the community's attention to this issue.
strength_weakness: strength_weakness: 1. The paper lacks quantitative metrics comparing the similarity between protected and unprotected images, such as PSNR, which would provide a more comprehensive evaluation of the method's impact on image quality.

2. The importance of the view-specific PGD angle has not been thoroughly explored. It would be beneficial to investigate whether the method is effective from different angles, not just the top view. Besides, the effectiveness of combined attacks on point cloud and triplane latent features should be explored to determine if they provide better protection.

3. Minor Issues: 
    a). Add a period at the end of "Quantitative comparison of perturbation strategies" in the Table 1 caption.
    b). Replace "no perturbation" with "not perturbed" (L255 P7).
    c). Correct the citation format "et al" to "et al." (L95 P3).
questions: questions: Please see the weakness.
limitations: limitations: The authors have adequately addressed the limitations
paper_summary: paper_summary: This paper proposed a new image cloaking approach, which adds adversarial noise on single-view image and makes TGS-based 3D reconstruction fail. This can be served as a watermark for protecting copyright image assets.
strength_weakness: strength_weakness: 1. The main weakness lies in the scope of this work is too limited. The motivation is to protect copyright images from unauthorized 3D reconstruction, however the work only targets at TGS-based reconstruction, which is too limited to have practical effects. I suggest the authors should at least do experiments on other single image 3D reconstruction works, for example LRM[1], Gamba[2] or LGM[3]. This work shows fair results on TGS, but TGS itself is not representative among 3D reconstruction works, and does not yield the best single image 3D reconstruction results.  In practice, we cannot assume unauthorized users will use which image-to-3D model, so it only makes sense when the watermarking / cloaking technique can effectively generalize to shield all those image-to-3D reconstruction algorithms.

2. Some typos.
Line 138, "preventing copyrighted"?


Reference
[1] LRM: Large Reconstruction Model for Single Image to 3D. ICLR 2024
[2] Gamba: Marry Gaussian Splatting with Mamba. https://arxiv.org/abs/2403.18795
[3] LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation. https://arxiv.org/abs/2402.05054
questions: questions: 1. How to specify the observation viewing direction? Is it arbitrarily selected?  How will the specified viewing direction affect reconstruction effects? (e.g. when the observation viewing direction is not good, will the algorithm fail?)

2. The example images shown in the paper use only one character or number for pre-defined patterns. If the watermark information is more complexed and needs multiple characters / numbers, how will the results be like?

3. Regarding weakness, can you discuss how this method can generalize to other single image to 3D reconstruction algorithms?  Is there any more generalized way to do watermarking/cloaking ?

4. In method description, the authors mentioned they use "a mask version of PGD", but details on how to generate the mask is not given in the text.
limitations: limitations: The author has discussed the social impacts. Overall this is a work for copyright protection, which does not have direct negative social impacts.

Comments: title: Thanks for your feedback
Thanks for your feedback 2
Thanks for your feedback 1
Thanks for your feedback
Thanks for your feedback
Response to the author's rebuttal
Thanks for your feedback
comment: Dear reviewer kd52,

We are very grateful for your recognition. We will integrate the results of the privacy budget into the final version of our paper based on your valuable feedback. 

3DGS-based single-view to 3D methods are novel mechanisms recently proposed. The properties of such mechanisms are still under exploration.  One possible explanation for this phenomenon could be that the feature of Tri-Plane is high-dimensional and implicit, while the geometric features (point cloud) are lower-dimensional and explicit. Explicit point clouds directly represent the attributes of the Gaussian (position of Gaussian). This discrepancy may make it easier to obtain appropriate perturbations when dealing with point clouds. We appreciate your insights, and we will incorporate all your valuable suggestions into our final paper.

Best regards,

Authors of #1185
Dear reviewer,

Thanks for your valuable feedback.

Our experimental findings indicate that explicit geometry features can be effectively utilized to protect ownership in GS-based tasks, despite TGS being a recent development. Due to the explicit feature of GS, GS-based image-to-3D methods inherently require similar explicit geometry features. This insight suggests that our approach can be extended to these methods, potentially enhancing security and ownership protection in single-image-to-3D applications and inspiring future research in this area.

We strongly agree with you on the concerns of the generalization ability. We will incorporate these additional experiments about our effectiveness on other GS-based approaches in the final version based on your valuable suggestions. We look forward to addressing your further concerns during this discussion.
We would like to clarify the misunderstanding of the settings of LGM and point cloud encoder.
LGM[1] and Gamba[3] do have an encoder to get point cloud.

### **Settings of LGM [1]**
For LGM, it takes 4 input views obtained from ImageDream/MVDream. Therefore, to verify the effectiveness of our method in undermining the generated 3D results, we directly optimize adversarial perturbations on these four input images.  LGM does have a **point cloud encoder** part to get the point cloud, as 3DGS needs a point cloud to represent the 3D scene. Specifically, please refer to Line 109 of LGM/core/model.py: https://github.com/3DTopia/LGM/blob/main/core/models.py 
```
109        pos = self.pos_act(x[..., 0:3]) # [B, N, 3]
110        opacity = self.opacity_act(x[..., 3:4])
111        scale = self.scale_act(x[..., 4:7])
112        rotation = self.rot_act(x[..., 7:11])
113        rgbs = self.rgb_act(x[..., 11:])
114        gaussians = torch.cat([pos, opacity, scale, rotation, rgbs], dim=-1) # [B, N, 14]
```
In our rebuttal settings, we directly target and perturbed the pos property of Gaussians, 
Line 109 pos = self.pos_act(x[..., 0:3]) # [B, N, 3],
which is the center of 3DGS. (i.e., point cloud). 


### **Adapting to LRM [2] and Gamba [3]**

For NeRF-based methods like LRM [2], as it does not require explicit geometry features, our method may not be suitable. We didn't experiment with Gamba as it has not been formally peer-reviewed.  However, it also has a point cloud encoder to estimate the point cloud (Line 130, position).

https://github.com/kyegomez/Gamba/blob/main/gamba_torch/main.py
```
130     # Position, opacity, color
131     position = self.position_layer(features)
132     opacity = self.opacity_layer(features)
133     color = self.color_layer(features)
```
Gamba is still required to estimate the point cloud. 

Thus, we do not need to change our motivation statement and algorithm description. The GS-based method must have a point cloud encoder to represent the 3D scene. 


[1] LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation. ECCV 2024.

[2] LRM: Large Reconstruction Model for Single Image to 3D. ICLR 2024

[3] Gamba: Marry Gaussian Splatting with Mamba for Single-View 3D Reconstruction. arxiv 2024
Thanks for your rebuttal, which clarifies my questions regarding viewing direction and multiple characters watermarking pattern. However, I still have significant concerns regarding the generalization capabilities of the current version of the work.


The authors suggest that their method can generalize to LGM by `simply adapting key elements`. However, the rebuttal materials do not provide sufficient details about the specific algorithmic adjustments required for LGM. LGM employs a fundamentally different architecture to achieve single image to 3D generation. Specifically, it first utilizes off-the-shelf models like ImageDream or MVDream to synthesize multi-view images, followed by an asymmetric U-Net architecture with cross-view self-attentions to construct Gaussian features. Given these foundational differences, it is unclear how the authors' approach could be seamlessly adapted to LGM[1]. For instance, the method described in the paper relies on a point cloud encoder from TGS and uses the Chamfer distance between two-point clouds as a loss to craft adversarial perturbations (c.f. Algorithm 1 and Figure 2). These critical components do not translate directly to LGM[1], as it does not incorporate point cloud encoders.  If authors wants to expand their method to other single image to 3D models, they would have to fundamentally change their motivation statement and algorithm description, which will make a huge modification to their current version of paper.
 

In the rebuttal, the authors mention that their method utilizes `geometry features in GS-based single views to 3D methods`, which I have reservations about this. Their initial claim is that the method is specifically tailored for Triplane Gaussian Splatting.  It is important to note that different approaches to single-image-to-3D conversion employ significantly diverse network architectures (e.g., LRM[2] uses transformers, Gamba[3] uses Mamba), as a result, their `geometry feature` space are highly distinct. Ensuring image copyright protection while considering generalization across these varied models is indeed challenging, and it may not be accurate to make such a broad generalization claim for their methodology design.

TGS[4] is a recently accepted paper in this field, but it is not a representative technique, and its reconstruction results are not among the best. The likelihood of it being widely used in real-world applications remains highly questionable. The key point here is that applying adversarial attacks to **a specific model** is too limited to be considered a broadly useful copyright protection method. Similarly, designing adversarial attacks tailored to a specific method may be too narrow in scope for a NeurIPS paper.

---
**Reference**
[1] LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation. ECCV 2024.

[2] LRM: Large Reconstruction Model for Single Image to 3D. ICLR 2024

[3] Gamba: Marry Gaussian Splatting with Mamba for Single-View 3D Reconstruction. arxiv 2024

[4] Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers. CVPR 2024
Comment: Dear Reviewer,

We are very grateful for your recognition. We will incorporate the experimental results into the main paper following your valuable suggestions.

Best regards,
Authors of #1185
Dear Reviewer,

We are very grateful for your recognition. We will integrate these results into the next version of our paper based on your valuable feedback.

Best regards,
Authors of #1185
I am satisfied with the authors' response and appreciate their effort to address my concerns regarding the effectiveness of noise perturbation depending on the privacy budget. 

Additionally, It is impressive that performance decreases as perturbation increases. 
While the authors notice that noise to the Tri-Plane do not produce the expected results, the reason for this phenomena remains unexplained. Although they have indicated this as future works, understanding the phenomena seems crucial in 3D reconstruction since learned Tri-Plane appears to also contain geometry information. Given these issue, I maintain my original score.
Thank you for your response. All of my concerns are addressed. The experiments of adapting to other methods should be incorporated into the main paper. The figures in the rebuttal for muli-char embedding are impressive. Thus, I will keep my original rating.
Thank you for your detailed response. My concerns are well-addressed. Specifically, this method exhibits notable effectiveness across various GS-based single-image-to-3D approaches, proving its broader applicability. Besides, from experimental results in the rebuttal, this paper also demonstrates that some complex patterns can also be efficiently generated for copyright protection. Considering its rebuttal and the two additional merits, I will increase my rating to 8.
Dear Reviewer, 

We are very grateful for your recognition. We will provide pixel-wise difference maps for protected and unprotected images in the final version based on your valuable suggestions.

Best regards,

Authors of #1185
Thank you for the thorough response. Your detailed explanations have resolved most of my concerns. Given the method's demonstrated extensibility and invisibility, I will be increasing my score. 

One additional suggestion: it may be beneficial to include a pixel-wise difference map between protected and unprotected images.