GS-Hider: Hiding Messages into 3D Gaussian Splatting

Authors: xuanyu  zhang
jiarui  meng
runyi  li
zhipei  xu
yongbing  zhang
jian  zhang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-04-27 08:22:35

Abstract: 3D Gaussian Splatting (3DGS) has already become the emerging research focus in the fields of 3D scene reconstruction and novel view synthesis. Given that training a 3DGS requires a significant amount of time and computational cost, it is crucial to protect the copyright, integrity, and privacy of such 3D assets. Steganography, as a crucial technique for encrypted transmission and copyright protection, has been extensively studied. However, it still lacks profound exploration targeted at 3DGS. Unlike its predecessor NeRF, 3DGS possesses two distinct features: 1) explicit 3D representation; and 2) real-time rendering speeds. These characteristics result in the 3DGS point cloud files being public and transparent, with each Gaussian point having a clear physical significance. Therefore, ensuring the security and fidelity of the original 3D scene while embedding information into the 3DGS point cloud files is an extremely challenging task. To solve the above-mentioned issue, we first propose a steganography framework for 3DGS, dubbed GS-Hider, which can embed 3D scenes and images into original GS point clouds in an invisible manner and accurately extract the hidden messages. Specifically, we design a coupled secured feature attribute to replace the original 3DGS's spherical harmonics coefficients and then use a scene decoder and a message decoder to disentangle the original RGB scene and the hidden message. Extensive experiments demonstrated that the proposed GS-Hider can effectively conceal multimodal messages without compromising rendering quality and possesses exceptional security, robustness, capacity, and flexibility. Our project is available at: https://xuanyuzhang21.github.io/project/gshider.

Main paper: 
 introduction: 
 As a frontier in computer vision and graphics, 3D scene reconstruction and novel view synthesis are crucial in fields such as movie production, game engines, virtual reality, and autonomous driving. Specifically, thanks to its high fidelity and fast rendering speeds, 3D Gaussian Splatting (3DGS) [21] has become a mainstream approach for 3D rendering. Considering that rendering a 3DGS is extremely costly, protecting the copyright and privacy of 3D assets should be a priority. As a widely studied technique in copyright protection, digital watermarking, and encrypted communication, steganography aims to hide messages like audio, images, and bits into digital content in an invisible manner. In its reveal process, it is only possible for the receivers with pre-defined revealing operations to reconstruct secret information from the container. Therefore, a natural idea arises: Can we design a steganography method tailored for 3DGS to protect the copyright and privacy of 3D scenes? ❑ (4) We conducted extensive experiments on the 3DGS dataset to demonstrate the security, robustness, fidelity, and flexibility of our method.2 Related Works 
3d representation: 
 The Neural Radiance Fields (NeRF) [35] marked a significant leap in novel view synthesis and multiview reconstruction. Efforts have focused on improving reconstruction quality [2, 3, 51, 4] , enhancing computational efficiency [50, 37, 48, 15, 43, 11] , and developing dynamic scene representations [42, 14, 6, 29, 12] . In recent advancements, 3D Gaussian Splatting (3DGS) [21] has emerged as a powerful method for reconstructing and representing 3D scenes using millions of 3D Gaussians. Compared to previous implicit representations such as NeRF, 3DGS offers significant improvements in both training and rendering efficiency. To further enhance the rendering performance of 3DGS, Mip-Splatting [62] achieves high-quality alias-free rendering at arbitrary resolutions by incorporating 2D and 3D filtering. Additionally, Scaffold-GS [30] introduces structured neural anchors, further enhancing the rendering quality of 3DGS from different viewpoints. The superior performance of 3DGS has expanded its applicability to a wide range of fields [54, 46] , including SLAM [20, 34] , 4D reconstruction [25, 31, 53, 57] , and 3D content generation [49, 58] .Steganography has been evolving over the decades [41, 7] . Thanks to advancements in deep learning, many deep steganography efforts aim to invisibly embed messages into containers and accurately extract them, including 2D images [64, 67, 1, 56, 61] , videos [36, 65, 32] , audio [26, 27, 8, 44] and generation models [52, 13] . Traditional 3D steganography approaches focus on watermarking explicit 3D representation such as mesh [38, 40, 55] via perturbing these vertices or transforming to the frequency domain. Meanwhile, Yoo et al. [60] aims to extract copyright from each 2D perspective, even when the complete 3D mesh is unavailable. Recently, watermarking implicit neural representations like NeRF have attracted increasing attention [33, 18, 24, 16, 28, 39, 68, 9, 47, 17] . For example, StegaNeRF [24] embedded images or audio into the 3D scene via fine-tuning the NeRF weights. CopyRNeRF [33] built a watermarked color representation and introduced a distortionresistant rendering strategy to ensure robust message extraction. WaterRF [18] introduced a deferred back-propagation technology with patch loss and resorted to discrete wavelet transform to enhance the fidelity and robustness of NeRF steganography. However, steganography for novel explicit representations 3DGS has not yet been explored.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: The paper discusses the limitations of the work performed by the authors in Sec. C.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
preliminaries: 
 As shown in Fig. 2 , 3DGS is an innovative and state-of-the-art approach in the field of novel view synthesis. Distinguished from implicit representation methods such as NeRF [35] , which utilize volume rendering, 3DGS leverages the splatting technique [59] to generate images, achieving remarkable real-time rendering speed. Specifically, 3DGS represents the scene through a set of anisotropic Gaussians, defined with its center position µ ∈ R 3 , covariance Σ ∈ R 3×3 which can be decomposed into scaling factor s ∈ R 3 and rotation factor q ∈ R 4 , color defined by spherical harmonic (SH) coefficients h ∈ R 3×(k+1) 2 (where k represents the order of spherical harmonics), and opacity α ∈ R 1 . Then, the 3D Gaussian can be queried as follows:EQUATIONwhere x represents the position of the query point. Subsequently, an efficient 3D to 2D Gaussian mapping [69] is employed to project the Gaussian onto the image plane:EQUATIONwhere μ and Σ separately represent the 2D mean position and covariance of the projected 3D Gaussian. P, W and J denote the projective transformation, viewing transformation, and Jacobian of the affine approximation of P, respectively. The color of the pixel on the image plane, denoted by p = (u, v), uses a typical neural point-based rendering [22, 23] . Let C ∈ R H×W ×3 represent the color of the rendered image where H and W represent the height and width of images, the rendering process is outlined as follows:EQUATIONwhere N represents the number of sample Gaussians that overlap the pixel p. c i ∈ R 3 and α i ∈ R 1 denote the color calculated from h i and opacity of the i-th Gaussian, respectively.Task Settings: Due to the slow rendering speeds of the implicit representation in NeRF, users often only access a few discrete rendered viewpoints online, rather than obtaining the entire NeRF weights. Consequently, NeRF trainers typically need to embed messages within the model weights and ensure that the same image or bit can be extracted from each rendered 2D viewpoint [24, 33, 18] . However, for 3DGS steganography, due to its real-time rendering capabilities, the trained point cloud files may be directly uploaded online. Therefore, our task setting is hiding messages during fitting the original 3D scene to create a container 3DGS, and then extracting embedded messages from it.The difference between ours and the NeRF steganography setup is that: In this section, we treat the hidden message as a single 3D scene for clarity. To achieve this task, we first review existing or some potential solutions.Original 3DGS: As shown in Fig. 2 (a), the original 3DGS renders RGB views from learned Gaussian points via a rendering pipeline, including projection, adaptive density control, and Gaussian rasterizer.The learnable attributes of i-th 3D Gaussian are represented asΘ i ={µ i , q i , s i , α i , h i }.Here, h i denotes the SH coefficients which are transformed to c i and represent the RGB color.To embed messages within a 3DGS, as plotted in Fig. 2 (b), an intuitive idea is to introduce another SH coefficient h ′ i to fit the hidden 3D scenes, namely learnable parametersΘ i ={µ i , q i , s i , α i , h i , h ′ i }.While this approach may achieve moderate fidelity for both the original and the hidden 3D scenes, it compromises security significantly. This is because stealers can easily detect the newly added h ′ i in the publicly available point cloud files and simply remove it. 3DGS+Decoder: Similar to StegaNeRF [24] , another intuitive approach is to add a decoder D m to forcibly ensure that the output RGB original views can reveal the corresponding views of the hidden scene. Thus, the learnable parameters are Θ i = {µ i q i , s i , α i , h i } and D m . However, considering that each Gaussian point has a specific physical significance, jointly optimizing the decoder could potentially compromise the optimization of the 3DGS, decreasing the fidelity of the original scene. Then, the scene and message decoder is adopted to decouple the rendered RGB scenes and hidden messages.Motivation: As depicted in Sec. 3.2, simply adding attributes or treating the 3DGS as a black box and jointly optimizing it with a decoder fails to meet the requirements for 3DGS steganography in security and fidelity. Thus, we aspire to have a coupling and decoupling process between the hidden and original information that are no longer presented independently. Inspired by the "Encoder+Decoder" structure in classical image steganography framework [1, 67] and combined with the characteristics of 3DGS, we find that the rendering and training process of 3DGS can be regarded as the "Encoder" for 3D scene fusion, and we can resort to additional deep networks as the "Decoder" to disentangle the original and hidden scene. Furthermore, exploring 3DGS attributes that can securely represent both the original and hidden scenes is challenging. As we have demonstrated in Sec. 3.2, merely adding a SH coefficient is unsafe. Inspired by [66, 25] , we find that rendering a high-dimensional feature map, as opposed to merely rendering an RGB image, can contain more information and provide greater confidentiality. Thus, we design a coupled secured feature attribute, a coupled feature rendering pipeline, and two parallel decoders to construct our GS-Hider.Defining Coupled Secured Feature Attribute: Different from h i in Fig. 2 (a), which has a fixed physical meaning, we define a more flexible attribute f i ∈ R M to replace h i ∈ R 48 as shown in Fig. 3 , where feature dimension M is arbitrary and adjustable. For i-th Gaussian, except for the center position µ i ∈ R 3 , scaling factor s i ∈ R 3 , rotation factor q i ∈ R 4 and opacity σ i ∈ R 1 , the coupled secured feature attribute f i ∈ R M serves to represent the color and textures of both the original 3D scene and the embedded message simultaneously. Defining f i has two significant benefits. 1) : It can effectively fuse the original scene and hidden scene via adaptive learning without the need to introduce a separate encoder or additional parameters. 2) : It is a safe and unified representation, which is impossible for stealers to distinguish which part of the feature represents the original 3D scene and which part represents the message scene.Constructing Coupled Feature Rendering Pipeline: Furthermore, we develop a coupled feature rendering pipeline shown in Fig. 3 . Similar to the original 3DGS, we follow the 3D Gaussian initialization via initial set of sparse points from SfM [45] and use the same projection strategy [69] to map the 3D Gaussians to the image plane in a given camera view. Unlike directly rendering an image, we do not need to convert h i into color component c i . Instead, inspired by Eq. 3, we use a coupled feature Gaussian rasterizer to directly render f i ∈R M into the coupled feature F coup ∈R H×W ×M .EQUATIONwhere N denotes the number of Gaussians that overlap the pixel p=(u, v). Specifically, the coupled feature rasterizer uses the tile-based rasterization technique, dividing the screen into 16×16 tiles, with each thread handling a single pixel. Different from the integration of c i ∈ R 3 in Eq. 3, the feature map F coup is directly rendered in a higher dimension M with greater information capacity. Finally, the coupled feature Gaussian rasterizer enables us to effectively blend the original scene and hidden message via the attribute f i , ensuring the hidden information remains confidential and high-fidelity.Disentangling Original and Message Scene: As plotted in Fig. 3 , after obtaining the rendered coupled feature F coup ∈R H×W ×M , we introduce the scene decoder D s to produce the rendered view of the original scene I pred ∈R H×W ×3 , and design the message decoder D m to extract the hidden Table 1 : Comparison of the PSNR(dB) performance of the original and hidden message scenes. We also report the average storage size of 3DGS point cloud files and the weights of decoders (if any). We are not directly comparing with 3DGS. In fact, 3DGS is the ideal upper limit of our performance. message scene M pred ∈R H×W ×3 as follows.EQUATIONTo be concise and ensure real-time rendering, both D s and D m consist solely of five stacked layers of convolution followed by ReLU activation functions. During deployment, D s will be public with the trained 3DGS point cloud file, while D m will be kept private as a special protocol, available only to users who are authorized to extract hidden information from the rendered feature F coup . In a nutshell, we use the attribute f i and rendering pipeline to couple the original and hidden information, and employ two decoders for disentangling the original and hidden scenes.To train our method, a training set of original scenes {I The learnable parameters of our GS-Hider are Θ i = {µ i , q i , s i , α i , f i }, D s , and D m . Finally, the training objective of our GS-Hider is defined as:EQUATIONEQUATIONwhere γ and β respectively denote the balancing weight of ℓ 1 loss and SSIM loss. Finally, our total loss is ℓ total = ℓ rgb + λℓ mes , where λ is used to trade off the optimization between the original scene and the message scene. Similar to 3DGS, during the optimization process, we employ an adaptive density control strategy to facilitate the splitting and merging of Gaussian points.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [No] Justification: The paper does not provide open access to the data and code.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental setup: 
 We conduct experiments on 9 original scenes taken from the public Mip-NeRF360 dataset [2] . The correspondence between the hidden and original scene are listed in Tab. 7. λ is set to 0.5 when hiding 3D scenes and set to 0.1 when hiding a single image. β and γ in Eq. 6 and Eq. 7 are respectively set to 0.2. The feature dimension M is set to 16. We conduct all our experiments on a NVIDIA RTX 4090 Server. Additionally, we modify the original CUDA rasterizer to support the rendering of feature maps of arbitrary dimensions. Considering that we are the first to implement 3DGS steganography, we compare the fidelity of our method with the original 3DGS and some intuitive approaches in Sec. 3.2. As shown in Tab. 1, compared to "3DGS+SH" and "3DGS+Decoder", our method achieves 25.817dB and 25.179 dB on PSNR for original and hidden scenes, far surpassing other intuitive methods. Compared to the original 3DGS, our method utilizes less storage space (half of the storage size) and incurs only a minimal decrease in rendering performance, while simultaneously possessing the capacity to conceal an entire 3D scene. Note that the performance of the original 3DGS is indeed the upper bound of our method. As plotted in Fig. 4 , it is evident that methods like "3DGS+Decoder" or "3DGS+SH" suffer from overlap artifacts between the recovered hidden and original scenes, resulting in limited fidelity and security. However, our method can distinctly reconstruct the two scenes without interference.To analyze on rendering speed of our GS-Hider, We test our GS-Hider on 9 public scenes using a NVIDIA RTX 4090 Server. The resolution of each scene is consistent with the experimental settings of the original 3DGS. The rendering time of the original scene via our GS-Hider is listed in Tab. 2. It can be observed that our method can achieve a rendering speed of 45 fps, which is much greater than the real-time rendering requirement of 30 fps. This proves the practicality and efficient rendering capabilities of our GS-Hider. The closer the curve is to the reference, the method is better in security.First, we claim that our coupled feature representation F coup is secure. This is because it is a highdimensional, complex, and chaotic feature, and only through our specific message decoder D m can the hidden message M pred be extracted from F coup . Fig. 6 visualizes three random channels of F coup ∈R H×W ×M and its corresponding original scene. It can be noticed that the geometric and texture of the feature map are almost consistent with the original scene, and no traces of the hidden information scene can be detected from it. It suggests that the coupled feature field hides messages more to the edges of the object and some imperceptible regions in the artifacts. Furthermore, to verify the security of our GS-Hider, we perform anti-steganography detection via StegExpose [5] on the rendered images of different methods. Note that the detection set is built by mixing rendered images of the original scene and the ground truth with equal proportions. We vary the detection thresholds in a wide range in StegExpose [5] and draw the ROC curve in Fig. 5 . The ideal case represents that the detector has a 50% probability of detecting rendered images from an equally mixed detection test, the same as a random guess. Evidently, the security of our GS-Hider exhibits a significant advantage compared to all competitive methods.To evaluate the robustness of GS-Hider, we have subjected the Gaussians to degradation using both sequential pruning and random pruning methods. Sequential pruning refers to pruning in ascending order of Gaussian's opacity, specifically removing Gaussians with lower opacity first. Random pruning, on the other hand, involves randomly pruning a proportion of Gaussians. Quantitative metrics are shown in Tab. 3. Sequential pruning has minimal impact on the performance of our model, and random pruning also shows minimal effect on the watermarked images. The results indicate that our method effectively withstands the degradation process.Ablation studies on key hyper-parameters of our GS-Hider are presented in Tab. 4. For the parameter λ, we observe that when λ=1, the GS-Hider could reconstruct the hidden scene with higher fidelity, making it more suitable for encrypted communication. Conversely, when λ=0.5, the GS-Hider was better at balancing the recovery of both the hidden and original 3D scenes. Regarding the feature channel M of F coup , we find that the optimal fidelity for both the original and hidden scenes is achieved when M was set to 16. Although the performance with M =8 is close to that of M =16, lower-dimensional features may result in hidden messages being more easily leaked, compromising security. Regarding the structure of the decoders D m and D s , we test various numbers of "Conv+ReLU" layers. Ultimately, we find that a configuration with five convolution layers allowed the GS-Hider to best balance the reconstruction accuracy of both the original and hidden scenes. Hiding multiple scenes into a single scene: To embed L hidden scenes into the original 3D scene, we need to modify the last convolution layer of D m to L × 3. Then, we jointly optimize L secret scenes and the original scene according to Eq. 7 and 6, which ensures each hidden scene, as well as the original scene, closely approximates the ground truth. To verify the effectiveness of our method for multiple 3D scene hiding, we conceal two groups of hidden scenes into two original scenes. As plotted in Fig. 8 , our method can store diverse results of 3D editing [10] within the original 3D scene, reducing the bandwidth load for transmission and presenting different content to different users. Additionally, the GS-Hider is capable of hiding two completely different scenes without interference, maintaining high fidelity. 
conclusion: 
 We propose a high-fidelity, secure, large-capacity, and versatile 3DGS steganography framework, GS-Hider. By utilizing a coupled secured feature representation with dual-decoder decoding, our method can conceal an image, one or multiple 3D scenes in a single 3D scene. To the best of our knowledge, GS-Hider is the first attempt to study 3D Gaussian splatting steganography, which can be applied for encrypted transmission, 3D compression, and copyright protection in 3D asserts. In the future, we will continue to enhance the fidelity and rendering speed of GS-Hider and expand its application scenarios, striving to advance security, transparency, and authenticity in the 3D community. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: The paper specifies all the training and test details necessary to understand the results in Sec. 4.1.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: The paper reports information about the statistical significance of the experiments, e.g. reconstruction accuracy in Sec. 4.2, security analysis in Sec. 4.3, and robustness analysis in Sec. 3.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legal to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [Yes]Justification: The paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 
licenses for existing assets: 
 Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: The creators or original owners of assets (e.g., code, data, models), used in the paper, are properly credited, and the license and terms of use explicitly are mentioned and properly respected.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA]Justification: The paper does not release new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: A Can our GS-Hider decode copyright from arbitrary 2D RGB viewpoint?Although this paper focuses on hiding and extracting messages from 3DGS point cloud files, our GS-Hider can also extract copyright from any 2D RGB perspective. In some cases, we may not have direct access to the complete 3DGS point cloud file or the rendering pipeline, but can only extract the copyright of the 3DGS from some sparse publicly available 2D views. Thus, similar to the task settings of [24, 33] , we design a rendering-resistant traceable watermarking strategy (RTWS) inspired by box-free watermarking approach [63] , which allows for the decoding of the copyright from any rendered 2D view by fine-tuning the GS-Hider and a newly added watermark decoder D w . Specifically, as plotted in Fig. 9 , we first use a pre-trained watermark encoder E w to embed a shared copyright watermark W cop into the training set of the original scenes {I(n) gt } T n=1 , obtaining watermarked view set {I ′(n) gt } T n=1 . Subsequently, we mix {I ′(n) gt } T n=1 and {I (n)gt } T n=1 in a 50% ratio and use this combined dataset to fine-tune the GS-Hider. To minimize the impact on the original rendering quality, we only alter the coupled feature attribute f i of the GS-Hider, without changing the positions or shape of each Gaussian point. Finally, we fine-tune the watermark decoder D w via the training pairs {I pred , I gt }, constraining it such that when I pred is input, the decoder outputs W cop , and conversely outputs a black image (W 0 ) when I gt is input.EQUATION)By adding watermarks and fine-tuning GS-Hider, we alter the training data domain, thereby causing the rendered images I pred to exhibit domain discrepancies compared to natural images I gt . By fine-tuning the D w , we enable it to detect this domain gap and decode exact copyrights. The network structure of E w and D w are similar to [64] . Note that the proposed RTWS in this section is not only applicable to GS-Hider, but also to original 3DGS, and even other 3D representations such as NeRF. To validate the effect of our GS-Hider for extracting copyright from RGB viewpoint, we use the proposed rendering-resistant traceable watermarking strategy (RTWS) to finetune our GS-Hider and a newly added watermark decoder D w . For simplicity, we choose the pre-trained GS-Hider that has embedded a single image. Tab. 6 reports the PSNR values of the original scene (PSNR S ), hidden message (PSNR M ), and the watermarked image extracted from RGB viewpoint (PSNR W ). It can be observed that our method is capable of extracting precise copyright watermark images from a 2D view with 39.927dB PSNR via D w , without sacrificing the fidelity of the hidden message and the original scene. To make the training view set of the hidden scene and the original scene correspond to each other, we use the trained 3DGS point cloud files and render them according to the viewpoints in the training set of the original scene to get the training view set of the hidden scene. To ensure that there are as many illegal views as possible in the training views of the hidden scene, we set the correspondence between the hidden and original scenes as listed in Tab. 7. We present the two main limitations of our GS-Hider and provide some potential improvements in the future works. 1) Compromised rendering quality: Since the feature attribute f i does not consider view-dependency compared to spherical harmonics, and we need to hide the secret scene while representing the original scene, our rendering quality is somewhat inferior to the original 3DGS. In fact, we inevitably need to make a trade-off between rendering quality and steganography capacity. However, our GS-Hider is a universal framework that can be integrated with the latest 3DGS variants, such as Mip-splatting [62] , to enhance rendering performance. Meanwhile, the current scene and message decoder designs are relatively simple. Integrating more efficient neural rendering and decoding designs (such as Scaffold-GS [30] ) can also help improve the overall rendering quality of the framework. 2) Decreased rendering speed: Due to the rasterization of high-dimensional features and network decoding, although we can still achieve real-time rendering, the rendering speed has decreased compared to the original 3DGS. However, we can easily improve rendering speed by pruning Gaussian points, reducing the dimension of feature attributes, and decreasing the number of convolution layers or feature kernels.To further verify the security of our GS-Hider, we randomly initialize the message decoder D m and use it to decode the rendered coupled feature. As shown in Fig. 10 , we find that using the wrong message decoder was completely unable to reconstruct the hidden scene, which further proves that it is difficult for unauthorized users to accurately decode our hidden scene. Hidden SceneTo verify whether our GS-Hider simply memorizes the hidden scene via the decoder, we conduct experiments of hiding two hidden scenes, as detailed in Sec. 4.6. This proves that our decoder is not storing or memorizing the secret information. Meanwhile, our message decoder is very lightweight with only 5 convolution layers. It only contains 0.465 M parameters, which is far from enough to memorize complex 3D scenes. In fact, the geometrical and structural information is mainly embedded in the coupled feature, and the role of the decoder is merely to extract and decouple the secret information, not to memorize the scene watermark. To show the role of our decoder, we further input the rendered coupled feature from another scene like 'playroom' to the message decoder that is trained to hide the scene 'bicycle'. The results are presented in Fig. 11 . We find that the rendered scene retains most of the geometric structure of the 'playroom' scene, with only some colors resembling those of the 'bicycle' scene. This indicates that our decoder itself cannot memorize secret information. First, the hidden scene information is concealed in the spatial high-frequency details of the coupled feature and some visually insensitive areas (such as artifacts, noise, and edges). The invisible hidden information in the coupled feature map will be amplified and decoupled by the message decoder, eventually forming an RGB hidden scene. We visualize the intermediate feature of the message decoder in Fig. 12 to illustrate this process. Second, the secret information is hidden in some redundant feature channels of the coupled feature field F coup . To prove this, we randomly set some channels in F coup to 0, and eventually find that the hidden decoder can not reconstruct the complete secret scene, as presented in Fig. 13 . This indicates that multiple feature channels are coupled and interact with each other, collectively storing the hidden information. Figure 13 : Visualization of the decoded hidden scene when some channels of the coupled feature F coup is randomly set to 0. Obviously, without some channels, the hidden scene cannot be fully decoded correctly.To verify the performance change of coupled secured feature attributes compared to spherical harmonic coefficients, we do not hide the message but only optimize the 3DGS attributes and scene decoder to fit the original scene. The PSNR, SSIM, and LPIPS of the rendered original scene are listed in Tab. 8. It can be found that without hiding any message, our method only has a PSNR reduction of 0.68 dB compared to the original 3DGS (listed in Tab. 1), which shows that our rendering performance is comparable to 3DGS. Meanwhile, as plotted in Table 1 , our storage size is only about half of that of 3DGS. By increasing the feature dimension M and the complexity of the decoder network, our rendering performance can be further improved and approach 3DGS. Finally, due to space limitations, we only put our PSNR results in Tab. 1. We also supplement all metrics of our GS-Hider on single 3D scene hiding in Tab. 9.To compare with the recent steganography method, we have tried our best to migrate the pipeline and decoding network of StegaNeRF [24] to the 3DGS steganography task. Specifically, we feed the The results are reported on Tab. 10. We find that GS-Hider is much better than 3DGS+StegaNeRF in terms of the reconstruction quality of hidden scenes, achieving 5.54dB improvement. This proves that our GS-Hider can effectively avoid the mutual interference between information hiding and scene rendering.To verify the generalizability of our framework, we realize a variant of GS-Hider based on Mipsplatting. Specifically, we retain the 3D smooth filter and 2D mip filter from Mip-splatting, only replacing the color attributes with high-dimensional features to fit the GS-Hider framework. Then, we conducted experiments on 3D scenes hiding on the mipnerf-360 dataset. The results are reported in Tab. 10. We also present some visualization results in Fig. 14 . This demonstrates that our GS-Hider is a universal steganography framework, not limited to specific 3DGS methods. Hidden SceneWe present more visualization results in Fig. 15 and Fig. 16 to demonstrate our effectiveness on 3D scene hiding and single image hiding. Moreover, we constructed an HTML file "./gshider/index.html" in the supplementary material to display some continuous 3D scenes. Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.Recovered ImageGuidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 


Decision: Before rebuttal most of the reviewers have concerns on the computational cost, etc. After rebuttal, all reviewers agreed that authors resolved their concern and the final rating of this paper is 1 strong accept and 3 weak accept. So AC decision to accept this paper.

Reviews: review: paper_summary: paper_summary: The paper introduces GS-Hider, a novel steganography framework for 3D Gaussian Splatting (3DGS). Protecting the security and fidelity of 3D assets while embedding information into transparent 3DGS point clouds is challenging, and the method addresses this by invisibly embedding 3D scenes and images into original GS point clouds. It employs a coupled secured feature attribute, scene decoder, and message decoder. Extensive experiments demonstrate its effectiveness in concealing multimodal messages without compromising rendering quality a lot.
strength_weakness: strength_weakness: 1) The rendering quality and speed are compromised a bit, but not much.
questions: questions: 1) It is interesting to understand where and how exactly your method hides the secret information. As you show in Figure 6, at first glance, it seems that the rendered coupled feature map only contains information about the original scene. Does it mean that the hidden message is encoded as (spatial) high-frequency details in the feature channels? Or maybe it learnt to hide information in last bits just like the least significant bit method? Are there any experiments that you conducted to investigate this more?
limitations: limitations: Limitations are adequately addressed.
paper_summary: paper_summary: The paper presents GS-Hider, a novel steganography framework designed for 3D Gaussian Splatting (3DGS). The framework enables the invisible embedding of 3D scenes and images into 3DGS point clouds, ensuring accurate extraction of hidden messages without compromising rendering quality. Extensive experiments demonstrate GS-Hider's effectiveness in concealing multimodal messages while maintaining exceptional security, robustness, capacity, and flexibility.
strength_weakness: strength_weakness: (-) The implementation of GS-Hider involves some techniques and may require substantial computational resources, limiting its accessibility and usability for some ordinary users without deep learning backgrounds.
(-) The comparison with existing methods is somewhat limited, as it primarily focuses on a specific type of 3DGS. It overlooks a broader range of state-of-the-art techniques in 3DGS. It may be beneficial for the author to consider implementing their methods in other 3DGS variants like [1] to highlight the advantages of the proposed method.

[1] Mip-Splatting: Alias-free 3D Gaussian Splatting.
questions: questions: Please see the weakness.
limitations: limitations: The authors have addressed the limitations in the paper.
paper_summary: paper_summary: The paper "GS-Hider: Hiding Messages into 3D Gaussian Splatting" proposes a steganography framework for 3D Gaussian Splatting (3DGS). GS-Hider embeds messages into 3D scenes by replacing spherical harmonics coefficients with a secured feature attribute and uses decoders to extract hidden and original scenes without compromising quality. Unlike traditional NeRF methods, 3DGS offers explicit 3D representation and real-time rendering. Experiments show GS-Hider maintains high fidelity, security, and robustness, making it suitable for copyright protection, encrypted communication, and 3D asset compression.
strength_weakness: strength_weakness: The authors discussed the limitations of their approach, which appear to be relatively minor.
questions: questions: I am curious about the scenario where an eavesdropper, let's say Eve, downloads Alice's publicly available model, renders it, and then trains her own GS. The purpose of Alice's steganography in Fig. 1 is to verify that the "Table" GS corresponds to her. However, if Eve can train her own "Table" GS, it seems Alice may not be able to protect her "Table" GS effectively. Could you please clarify how this concern is addressed?
limitations: limitations: Please refer to the questions.
paper_summary: paper_summary: This paper presents GS-Hider, a novel framework for steganography in 3D Gaussian Splatting (3DGS) models. The key innovation is a coupled secured feature attribute that replaces the original spherical harmonics coefficients, allowing the embedding of hidden 3D scenes or images into the original scene without compromising rendering quality. The framework uses a scene decoder and a message decoder to disentangle the original and hidden information. The authors demonstrate the effectiveness of GS-Hider across various experiments, showing its ability to hide multiple 3D scenes or single images while maintaining high fidelity, security, and robustness.
strength_weakness: strength_weakness: Main weakness:
* The main weakness is the additional computational overhead introduced by using the scene decoder. Gaussian splatting significantly simplifies the neural rendering paradigm of NeRF into a point vector + rasterization approach, achieving very fast rendering - this is one of the most valued aspects of GS. However, the introduction of the scene decoder greatly undermines this advantage. This makes such an improvement meaningless. I am confident in this inference because the authors do not seem to report any FPS-related metrics, nor do they compare the FPS increase with the original 3DGS.
* The decoder is trained per scene, rather than being a generalizable decoder, which suggests it may be "memorizing" scene watermarks. This can be inferred from the issues involved in training a decoder for each scene and hiding only a single scene per scene. At the same time, the geometrically inconsistent signals present in the cover scene and hidden scene are well accommodated together, further indicating that the signals are to some extent being "memorized" in the decoder. Therefore, I speculate that the capacity of the decoder is not small. This further corroborates the aforementioned weakness.

Some other weakness:
* While the empirical results are strong, there's a lack of theoretical justification for why the coupled feature representation works so well.
* Comparison to recent work: The paper could benefit from comparing against more recent steganography methods, particularly those designed for other 3D representations that might be adaptable to 3DGS.
* Generalization: The experiments are limited to a single dataset (Mip-NeRF360). It would be valuable to see how the method performs on a wider range of 3D scenes and different types of hidden information.
questions: questions: Main issues that could be considered in rebuttal：
* what's the added inference time overhead due to using the scene decoder?
* the fps metrics for all experiments, so that we can see the impact on the practical usage. If the rendering speed is affected, the improvement about privacy is not useful.
* try to prove that the hidden scene is not simply memorized by the decoder. maybe trying to memory more than one hidden scenes or curating a general decoder that can be applied to not only one scene can be convincing

Other issues:
please see the weakness
limitations: limitations: The authors provide a brief discussion of limitations in Section B of the appendix, acknowledging issues such as the lack of view-dependency and slightly reduced rendering quality. They also mention future work directions, including enhancing model expressiveness and extending to tasks like tampering detection. However, this discussion could be expanded to more thoroughly address potential failure modes or edge cases of the proposed method.

The broader impacts section (A.2) touches on both positive applications (e.g., copyright protection) and potential risks (sensitive data concerns). However, a more in-depth exploration of potential misuse scenarios and mitigation strategies would strengthen the paper's ethical considerations.

Comments: title: Thank Reviewer koxG for recognizing our work
Thank Reviewer X8jQ for recognizing our work
Thank Reviewer crVb for recognizing our work
Thank Reviewer S9BH for recognizing our work
Thanks
Looking forward to discussions with reviewers
comment: Dear Reviewer koxG:

We are very grateful for your recognition of our work. We will include your valuable suggestion in the revised version. Your insights have significantly contributed to the improvement of our work.

Best Regards,

Authors of #902
Thank you for your response. It has resolved my main concerns. Regarding the adversarial scenario we discussed, I believe revisiting it would be beneficial to ensure preparedness.
Dear Reviewer X8jQ:

Thank you for your response and for recognizing our work. We will include the content from the rebuttal in the final version.

Best Regards,

Authors of #902
I thank the authors for their response. I will keep my score.
Dear Reviewer crVb:

We sincerely appreciate your prompt response, valuable suggestions, and recognition of our work. We will include the additional experiments from the rebuttal and provide detailed explanations in the final version to make our paper more rigorous and complete.

Best Regards,

Authors of #902
Thank you for your response. It addresses my concerns regarding potential added overhead of inference time. So I raise the rating. I recommend that the revision could include the supplied results in your rebuttal.
Dear Reviewers,

Authors have provided their rebuttal. Could you please take a look to see whether your concerns are addressed or not? Thanks!
Dear Reviewer S9BH:

Thank you for your prompt response and for acknowledging our work. We sincerely appreciate your valuable suggestions and assure you that we will include the additional experiments and explanation from the rebuttal in the final version.

Best regards,

Authors of #902
Thanks for your detailed response, which has addressed my concerns. After reading the rebuttal and other reviews, I believe this work has practical applications in copyright protection of 3D assets. Meanwhile, the author implemented a variant of GS-Hider based on mip-splatting in the rebuttal, verifying that the proposed method is a general framework. Therefore, considering the novelty of this paper in 3D steganography, good rendering quality and real-time rendering speed, I decide to raise my score to 8. However, please also remember to address my concerns in your final version, especially those parts that you have promised.
Dear Reviewers:

We appreciate the time you dedicated to reviewing our work and your recognition of our work. Regarding the concerns you raised, we have provided explanations in our responses.

We would like to ensure that your concerns have been adequately addressed. If there are any aspects of our work that remain unclear to you, please don't hesitate to let us know.

Thank you for your dedication!

Best regards,

Authors of #902