Privacy-Shielded Image Compression: Defending Against Exploitation from Vision-Language Pretrained Models
Authors: Xuelin Shen
Jiayin Xu
Kangsheng Yin
Wenhan Yang
Venue: ICML 2025 poster (2025)
Keywords: Privacy Safeguard
Backdoor Attack
Vision-Language Pretrained Models
Learned Image Compression
Primary area: applications->computer_vision

Abstract: The improved semantic understanding of vision-language pretrained (VLP) models has made it increasingly difficult to protect publicly posted images from being exploited by search engines and other similar tools. In this context, this paper seeks to protect users' privacy by implementing defenses at the image compression stage to prevent exploitation. Specifically, we propose a flexible coding method, termed Privacy-Shielded Image Compression (PSIC), that can produce bitstreams with multiple decoding options. By default, the bitstream is decoded to preserve satisfactory perceptual quality while preventing interpretation by VLP models. Our method also retains the original image compression functionality. With a customizable input condition, the proposed scheme can reconstruct the image that preserves its full semantic information. A Conditional Latent Trigger Generation (CLTG) module is proposed to produce bias information based on customizable conditions to guide the decoding process into different reconstructed versions, and an Uncertainty-Aware Encryption-Oriented (UAEO) optimization function is designed to leverage the soft labels inferred from the target VLP model's uncertainty on the training data. This paper further incorporates an adaptive multi-objective optimization strategy to obtain improved encrypting performance and perceptual quality simultaneously within a unified training process. The proposed scheme is plug-and-play and can be seamlessly integrated into most existing Learned Image Compression (LIC) models. Extensive experiments across multiple downstream tasks have demonstrated the effectiveness of our design.

SSheeleed Image Compression: Defending Against Exploitation from Vision-Language Pretrained Models
Xuelin Shen
Jiayin Xu
Kangsheng Yin
Wenhan Yang'
Abstract
The improved semantic understanding of vision- language pretrained (VLP) models has made it increasingly difficult protect publicly posted images from being exploited by search engines and other similar tools. In this context, this pa- per seeks to protect users' privacy by implement- ing defenses the image compression stage to prevent exploitation. Specifically, we propose a flexible coding method, termed Privacy-Shielde Image Compression (PSIC), that produce bit- streams with multiple decoding options. By de- fault, the bitstream is decoded to preserve satis- factory perceptual quality while preventing inter- pretation by VLP models. Our method also re- tains the original image compression functionality. With customizable input condition, the proposed scheme can reconstruct the image that preserves its full semantic information. A Conditional La- tent Trigger Generation (CLTG) module is pro- posed to produce bias information based on cus- tomizable conditions to guide the decoding pro- cess into different reconstructed versions, and an Uncertainty-Awer Aware Encryption- -rrrittto--Oiee (UAFO) optimization function designed to leverage the soft labels inferred from the target VLP model's uncertainty on the training data. This paper fur- ther incorporates an adaptive multi-objective op- timization strategy to obtain improved encrypt- ing performance and perceptual quality simulta- neously within unified training process. The proposed scheme plug-and-play and can be seamlessly integrated into most existing Learned Image Compression (LIC) models. Extensive ex- periments across multiple downstream tasks have demonstrated the effectiveness our design.
Enpux
Fill
Enerypted
Enerypeed Cadlag
3E
people perpare
table fill police
recarding mansi..
dnajaic (Feage
Captionhng
Figure f. In the default protected coding mode, our compression model protects images by preserving content similar to the original, while concealing machine- peerevvve semanties. Additionally, the model can encode images in a way that maintains both critical pixel fidelity and semantics.
1. Introduction
In recent years, the rapid advancement of large-scale Vision- Language Pretrained (VLP) models has transformed tra- ditional task-specific approaches, significantly improving the performance across range of vision tasks. State-of- the-art VLP models, such Contrastive Language-Image Pre-training (CLIP) (Radford et al., 2021), A Large-scale ImaGe and Noisy-Text Embedding (ALIGN) (Jia et al., 2021), and InstructBLIP (Dai et al., 2023), demonstrate excellent performance in general sense for visual informa- tion understanding. They perform well in tasks including not only image text retrieval (Fang al., 2021; Ma et al., 2022; Luo et al., 2022; Yan et al., 2023), image captioning (Chen et al., 2015), visual question answering (Antol et al., 2015), cross-modality grounding (Peng et al., 2023b), but also conventional vision tasks including image classification (Fu et al., 2022; Peng et al., 2023a; Abdelfattah et al., 2023), facial attribute analysis (Yang et al., 2022: Zhou al., 2024). and image segmentation (Xu et al., 2022; Chen al., 2023).
'Equal contribution 'Pengcheng Laboratory Guangdong Lab- oratory of Artificial Intelligence and Digital Economy (SZ) 'Shenzhen University. Correspondence to: Wenhan Yang Sraanwh@@cl.ac.oom
VLP models are highly effective capturing semantic in- formation, but this strength also comes with risks. Their powerful ability to analyze semantics raises concerns about privacy and data security. Images shared on community platforms can be easily understood by these models, making
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Privacy-Shiclded Image Compression: Defending Against Exploitation from Vision-Language Pretrained Models
it hard to stop them from being indexed by search engines or reused as open data without the user's permission. For instance, in (Johan Modin, 2022), the authors demonstrate the feasibility of tracking the trajectory of a specific vehicle or person from a gallery of surveillance videos by provid- ing only a textual description to the VLP model. Similarly, by incorporating these powerful VLP models, search en- gines could more easily retrieve relevant images without the need for handcrafted labels. Under these cireumstances, there is significant potential to explore privacy protection approaches in the context of defending against VLP models. This emphasizes the need to protect user data and prevent its misuse.
patterns capable of significantly misleading the targeted VLP model; and the full version, which, with customiz- able input condition (e.g., keywords), provides the image's full semantic information. Moreover, to fulfill these mu- tually exclusive requirements within a single network, the proposed PSIC incorporates an adaptive multi-objective op- timization strategy, where the model alternates between training under the rate-distortion and rate-eneryption crite- ria, guided by the corresponding mode indicator. Besides using the intuitive optimization function that minimizes the similarity between the reconstructed image and its corre- sponding text counterpart (interfering with the fitting to certain labels), we further propose an Uncertainty-Awart Encryption- Oriennted (UAEO) optimization function. The proposed UAEO automatically identifies uncertainty among the image-text pairs using the Dempster-Shafer Theory, and to maximize the similarity between cross-modal matches with higher matching uncertainty (forcing fitting to uncer- tain wrong labels) during the training process. The proposed PSIC scheme is blug-and-play and can be seamlessly inte- grated into most existing learned image compression (LIC) models. Extensive experiments demonstrate that the pro- posed scheme effectively conceals privacy information by misleading VLP-based applications, such as text-image re- trieval and image captioning, as well as common computer vision tasks like image classification and facial attribute analysis, all while maintaining the same rate-distortion per- formance as baseline LIC models. We summarize the main contributions as follows:
An intuitive idea is to embed certain information into image coding (compression or transcoding) that does not affect visual perception, enabling universal and efficient way to protect the content of the image. A related research direction focuses on the vulnerability of deep learning models to spe- cific noise patterns. In particular, existing backdoor attack schemes ( YYu et al., 2023; 2024) follow a similar approach by embedding invisible triggers into input images. These triggers are designed to degrade compression processes and transform them into adversarial attack patterns, effectively misleading downstream machine vision models. Directly applying these methods to address the proposed new prob- lem faces two key challenges: 1) In the context of designing privacy-ppooe criaacy-rotecctng compression networks, injecting triggers at the input stage inevitably reduces flexibility and com- pression efficiency. Specifically, this backdoor attack is an irreversible process, leading coding practice that re- quires separate encoding steps-one with trigger injection and one without-in order generate distinct bitstreams for attacking and normal purposes. 2) Since an ideal privacy- preserving coding could support both visual perception and machine recognition (may under different coding modes), the challenge lies in efficiently coding both types of infor- mation and activating them under specific conditions. This issue has not been explored in previous methods.
We propose flexible image compression network that can adapt to user needs by controlling whether images can be accessed by downstream VLP models, all while maintaining good perceptual quality.
We propose a Conditional Latent Trigger Generation (CLTG) module and an adaptive multi-objective op- timization strategy to enable single bitstream to be decoded into two versions with mutually exclusive ob- jectives.
In this paper, we focus on Image Compression problem un- der the consideration of privacy protection problem and pro- pose a novel Privacy-Shieldes Image Compression (PSIC) framework. 1 This framework builds on a formulation sim- ilar to backdoor attacks and introduces new innovations to significantly improve coding flexibility and efficiency, while also ensuring compactness, human perception compat- ibility, and encryption requirements, among other valuable attributes. In detail, the proposed PSIC enables a single bitstream to be decoded into two distinct versions through a conditional trigger injection module takes the mode indica- tor (encrypted or full) as an additional input: the encrypted version, produced as default, which embeds invisible attack
We introduce an Uncertainty-Aware Eneryption- Oriented (UAEO) Optimization Function that exam- ines the uncertainty within the target model's prior knowledge, thereby enhancing robustness and increas- ing the success rate of misleading the target VLP model.
2. Related Work
2.1. Visual Language Pre-trained Models
VLP involves a new machine learning paradigm trained on large datasets of images and corresponding text, aiming to develop generalized models that understand content integrat- ing both visual and linguistic information. Existing VLP
"Code available at http:://giiaabboomm///iiixxsssss
2

---

Privacy-Shiclded Image Compression: Defending Against Exploitation from Vision-Language Pretrained Models
models can be mainly categorized into single-stream and dual-stream architectures (Du et al., 2022). The former (Li et al., 2019) leverages single network, e.g., transformer, to fuse both image and text features, facilitating downstream tasks. In contrast, the latter utilizes separate encoders for images and texts. A representative example is CLIP (Rad- ford et al., 2021), which employed distinct image and text encoders align embeddings through a contrastive learning approach, thereby learning a shared representation between images and texts. CLIP has demonstrated excellent perfor- mance in a variety of tasks such as zero-shot classification and image captioning, Recent approaches have integrated the strengths of both single-stream and dual-stream archi- tectures to enhance performance. For instance, BLIP (Li et al., 2022) introduced a multimodal mixture of encoder- decoder architecture along with a captioning and filtering mechanism to tackle vision- vision-language understanding and generation tasks. Building upon this, BLIP-2 (Li et al., 2023) incorporated a lightweight Querying Transformer (Q- Former) to align image and text features, thereby improving multimodal processing capabilities and computational effi- ciency. Further advancing this line of research, InstructBLIP (Dai al., 2023) integrated an instruction-aware feature extraction method, enabling the model to derive specific representations from images based on varying instructions.
image compression techniques.
2.3. Backdoor Attacks
Deep learning has been shown to be highly susceptible to backdoor attacks. These attacks typically involve modify- ing a model's parameters or input data, with the goal of poisoning training samples by injecting triggers to embed malicious patterns (Gao et al., 2023). When the model encounters a specific trigger during inference, it produces incorrect outputs. Due to the malicious nature of backdoor attacks, they have garnered significant attention across vari- ous domains, such as BadNet (Gu al., 2017), SIG (Barni et al., 2019), and SSBA (Li al., 2021). In the field of Vision-Language Pretraining (VLP), various strategies are employed for attacking purposes. For instance, targeting CLIP, Yang et al. (Yang et al.. 2023) adjusted the encoder to maximize the cosine similarity between image and text embeddings, leading to misclassification in image-text re- trieval tasks. Meanwhile, Bai al. (Bai et al., 2024) utilized prompt learning techniques to train learnable parameters that distort the latent representations generated by the encoder. Liang el al. (Liang et al., 2024) proposed dual-embedding guided framework for backdoor attacks, characterized by its stealthiness against backdoor detection due to the subtle parameter changes it induces.
2.2. Learned Image Compression
While these attack methods have demonstrated significant effectiveness, they operate under the assumption that input images pristine. However, in practical scenarios, images are typically compressed before being delivered to the target model. Overlooking the compression process can inevitably lead to unsatisfactory attack performance. In this context, Yu et al. (Yu et al., 2023; 2024) investigated how specific trigger patterns can be disrupted during the compression process, leading to compressed images that mislead down- stream machine analytic tasks.
In the past decade, significant progress has been made in the field of LIC. Ballé et al. (Ballé et al., 2017) were the pi- oneers in proposing the first end-to-end image compression network based on Variational Autoencoders (VAE) (Kingma & Welling, 2013). This was followed by the introduction of prior networks and context-based entropy models, which sig- nificantly improved the rate-distortion performance, Along this research direction, many efforts have been dedicated to further improving the entropy module. For example, Guo et al. (Guo et al., 2021) introduced enhanced entropy coding with global contextual information, while the chessboard context model and channel context were introduced in He et al. (He et al., 2021) and Minnen el al. (Minnen & Singh, 2020), respectively, to reduce the computational complexity of entropy encoding. Furthermore, Jiang al. (Jiang al., 2023) combined local spatial, global spatial, and channel context to propose refltiereference entropy model, achiev- ing state-of-the- performance at the time. Beyond entropy models, other research works have focused on improving compression performance by adopting more powerful back- bone networks. Notably, Rippel et al. (Rippel & Bourdev, 2017) proposed GAN-based image encoder that improved reconstruction quality through adversarial loss. Recently Yang et al. (Yang& & Mandt, 2024) introduced an end-to-end optimized compression framework based on diffusion gen- erative models, further advancing the development of lossy
3. Privacy-Shielded Image Compression
3.1. Overview and Motivations
As mentioned above, our method Privacy-Shicldee Image Compression (PSIC) is designed based on the critical mo- tivation: supporting both visual perception and machine recognition, potentially under different encoding modes through coding and integrating these two types of infor- mation and activating them under specific conditions. In general, PSIC is characterized by its flexible conditional representation capacity, allowing a single bitstream to be decoded into two different versions: the full version, which preserves satisfactory perceptual quality and full seman- tic information, and the encrypted version, which contains nearly invisible adversarial attack patterns that significantly mislead downstream VLP or machine vision models. More-
3

---

Privacy-Shiclded Image Compression: Defending Against Exploitation from Vision-Language Pretrained Models
Full
Decoder
CLTG
L
Embd
Be
ResBlock 8g
B
faec
Input x
Linear
CLTG
Relu
Conv
Encoder
Decoder
ResBlock
Encrypted &
Linear
CLTG
Conv
Linear
8e
Luc
fatec
(b)
hrre people prepare table tull food with police kackuud
snowy landscape with againsl a backdrap of unique, glowing archilecture.
winter ciothing,
Uncertainty
Text Embeddings
biological species with pup along a city.
sandcastle beach on bright sky.
Evidence Extractor
CLIP
Image Embeddings
Training Sample
Training Batch
Figure 2. (a) Compressing pipeline of the proposed Privacy-Shiie Shielded Image Compression (PSIC). (b) Details of the proposed Conditional Latent trigger Generation (CLTG) module. (c) Intuitive illustration the introduced Uncertainty-Awar Encryption- Eneryption-Orieened Oriented (UAFO) optimization function.
over, under the default setting, only the encrypted version is produced. The full version is generated only when users provide customizable conditions (e.g., keywords) to the de- coding side.
ú and jointly fed the decoder D(-), generating the corre- sponding version tailored the user's requirements,
fe DÚÚ Te): ã = (Ú)TT))
(3)
where f and ê denote the encrypted version and ver- sion, respectively.
The overall framework is illustrated Fig. 2 (a). At the encoding side, the input image & would be first fed to an encoder E(-) obtain its compact latent representation j.
In realizing the challenging coding pipeline mentioned above with satisfactory compression efficiency, three in- novative designs play key roles: /) the Conditional Latent Trigger Generation (CLTG) module, which enables the generation of different triggers through single module, in- fluences the decoding process to achieve mutually exclusive objectives; 2) the Uncertainty-Aware Encryption-Orient (UAEO) optimization function, which leverages the CLIP model the attacking target, being capable of automati- cally identifing low-confidence cross-modal matches in the training via Dempster- Shafer Theory, enhancing robust- ness and the success rate of attacks; 3) the adaptive multi- objective optimization strategy automatically balances the distinct requirements of encrypted and full version recon- struction, instead relying on an empirically decided ap- proach, leading to improved compression performance. In
t Fxx).
(1)
An entropy model Q(-) is leveraged to produce the binary bitstream of ý and provide its bitrate indices Moreover, alongside the bitstream, the customizable condition 3 must also be provided by the users and transmitted to the decoding side. For clarity, we denote the 3. and By as the indicators for the encrypted version and full version, respectively.
At the decoding side, the 8 Bye would be fed to trigger generator 7(.), obtaining corresponding feature-wise triggers F E [Te, Tyl.
Te T(3e), Tf T(By)-
The triggers would be injected the latent representation

---

Privacy-Shiclded Image Compression: Defending Against Exploitation from Vision-Language Pretrained Models
the following subsections, the CLTG module, the UAEO optimization function, and the adaptive multi-objective opti- mization strategy will be detailed in Subsec. 3.2, Subsec. 3.3 and Subsec. 3.4, respectively.
(Dempster, 2008) of evidence, which combines evidence from different sources with a degree of belief (also known as the belief function), is adapted to obtain pair uncertainty.
To be specific, consider the input image 2s, and text ty within a batch of size K, i.e., i, E [1, K], where = j indicates they are labeled as a pair. For a certain image-text pairs (zi, ty), we employ the evidence extractor proposed in (Qin al., 2022) (--) obtain their evidence esj.
3.2. Conditional Latent Trigger Generation Module
Compared to existing LIC-oriented backdoor attack ap- proaches, where the trigger generation process considers only the attacking purpose, PSIC faces a more significant challenge: it aims to reconstruct a single bitstream into two versions with mutually exclusive objectives. To ad- dress this, we propose a progressive trigger injection strat- egy that deploys series of CLTG modules with indepen- dent parameters at each decoding block, as shown in Fig. 2 (b). The deployed CLTG modules take the mode indicator 3 E {Be, 8y} } as input and adaptively produce bias informa- tion at different representation stages.
Cij = ( tz) = eap tanb (Cip(((
(5)
where Clip(s denotes the cosine similarly in the CLIP
feature space, s E (0, 1) is scaling factor.
Accordingly, the image-to-tess evidence vector within a batch eqat = [ea e:2; eik] would be obtained, as well as the test-to-image evidence vector ef2i [e1j E2j ..... eKy]
Then, we calculate the cross-modal bidirectional evidence vector via,
In particular, within the CLTG module, the mode indicator 3 would be first fed in to 2-layer MLP, obtaining the corresonndigg conditional bias feature fa- This bias feature, Ss. is then fused with the image feature to bias the decoding process towards its indicated preference,
i2t + e =
b
=lea- E
121
exxl.
(6)
Thus, the corresponding Dirichlet distribution strength is parameterized by [c] + 1, + 1,..., cb
CK + 1] with the k
SCLTG = JaWa + Sdees
(4)
distribution strength approximated by دe= (el) + +1. And
the uncertainty mass Wiy of the image- mext pair (2,, tz) can be calculated as
where fCLTG and fdee denote the output and the input features of the CLTG module respectively, Wa is the feature element-wise weighting map with d denoting the channel depth aligned with the input features.
Uij = K Σj (ely + +1)
3.3. Uncertainty-Aware -rrreption-OOieented (UAEO) Optimization Function
Subsequently, for the i-th image Is, the corresponding prompt t, with highest uncertainty could be obtained via,
In the proposed PSIC, the decoded encrypted version is expected to contain adversarial patterns that significantly mislead the target CLIP model. To this end, a simple strat- egy involves setting an objective function for the encrypted version é. reconstruction by minimizing the cosine similar- ity between fe and its textual prompt t in the CLIP feature space. However, this approach is inevitably not capable of providing stable and efficient optimization path, as it does not identifyy specific target during the training pro- cess. Moreover, acknowledging that the existing multimodal training data is often quite raw, with many datasets collected from the Internet, it inevitably contains image pairs that are not well aligned. Directly applying the naive objective function may result in issues with robustness and efficiency. Given the above considerations, we are motivated to capture and leverage the uncertainty within the CLIP model's ''s prior knowledge, as illustrated in Fig. 2 (c). In general, the en- cryption-oriented optimization involves in maximizing the similarity between cross- modal matches with higher match- ing uncertainty, while decreasing the similarity with the rest within a training batch. The Dempster-She Shaffer Theory (DST)
in arg min Uins
(8)
Thus, the encryption oriented optimization function Lve can be formulated by,
exp ErD(CLIP(((==)).gttt)))
Luc(x;) = -log
(9)
Σ (LP(((L)((()))))(()))))
3.4. The Adaptive Multi-Objective Optimization Strategy
The proposed PSIC is designed to obtain a compuct latent representation from the encoding stage, which is then adap- tively decoded into two versions that meet the encryption and perception requirements, respectively. Given the sig- nificant challenges posed by the mutual exclusivity of the three objectives, we develop a two-stage training strategy. The first stage focuses on compacteess while the second stage addresses the divergent representations tailored for encryption and perception requirements.
5

---

Privacy-Shiclded Image Compression: Defending Against Exploitation from Vision-Language Pretrained Models
Rgzt for di
Accunsyy for Image dhoificaio
Arzuray stmibete analycs
3.5
6.54
6.4
Miltargee
0.4
8:30
Riued plave
0:3
02
635
0:30
0.12
6.25
023
145
0:55
625
635
E45
0.15
635
0.3
615
8:5
0.33
Serdop
Serilpp
Rok (hap)
(a)
(b)
(4)
ASR for difeveri taaks
Blurp 26 Iaage OpToOning
CIDP: Imag exptiening
SPICE Sy Image cepticinng
6.70
Mottt plant
Therthi yoar
0:0
013
6:3
0:35
645
015
6.5
6.11
6.45
Bale (po)
3ac
(f)
(g)
(h)
Figure 3. Performance comparisons regarding the four employed downstream tasks.
First Stage, As for the first stage, the encoder Eg, entropy model Qs, and the conditional decoder D&,, parameterized by 0, o, vù, respectively, are jointly optimized by alternating between rate-encryption and rate-perception criteria. In particular, we split the training iterations within one epoch into two sessions, and the corresponding training process can be formulated by:
4. Experiments
During our experiments, we compare LIC networks both with and without our PSIC implementation assess per- ceptual quality and encryption effectiveness. Additionally, we employ cutting-edge LIC-oriented backdoor attack method to demonstrate the superiority of the proposed PSIC. Furthermore, we conduct comprehensive ablation studies to highlight the effectiveness of our specially designed mod- ules.
$ 111:0, 5. = arg min (0.v.0) (FEX)
Σ
A. A2Lr, (F((()))
+ Q1)
4.1. Experimental Setup
Benchmark. Four downstream tasks are employed for com- prehensive evaluation, including two cross-modas applica- tions (image-text retrieval and image captioning) and two common machine vision tasks (image classification and facial attribute analysis).
S 112:00 arg
Σ A (0,0.0) (xEX)
+ LDc Be))) + (z)
where À is a Lagrange parameter to obtain the ICM models at different compression levels.
In particular, image-text retrieval, image classification, and facial attribute analysis are performed directly by the tar- get model, CLIP (ViT-B/32), during training. Meanwhile, image captioning is conducted on the unseen BLIP-2 (OPT- 2. 2.7b) (Li et al., 2023), to demonstrate the generalization capacity.
Second Stage. In the second stage, the parameters of the encoder and entropy model are frozen, allowing the focus to shift towards the divergent representations needed to fulfill the encryption and perception requirements. Specifically, the iterations within each epoch are alternately optimized with the following criteria,
Image-text retrieval: Performances for both the image- to-text (i2t) and text-to-image (t2i) tasks are evaluated using Recall @ (R@1) 1).
S S2-1:0 arg min (0))
Σ (xEX)
La(x, (s
Image classification: We employ 5,000 images from the ILSVRC 2012 dataset (Deng et al., 2009) (ImageNet- Ik). Performance is assessed using Top-1 accuracy.
S2-2: arg min
Σ
LFc(DD(Fooo),
(V)
(zEX)
6

---

Privacy-Shiclded Image Compression: Defending Against Exploitation from Vision-Language Pretrained Models
PRWR
comparison. Specifically, BAvAFT involves injecting trig- gers into the input images with the expectation that the reconstructed images will significantly mislead downstream models. We implemented the BAvAFT on the same back- bone LIC network, training it from scratch using the same training set and targeting the Vision Transformer model ViT-B/32, consistent with our PSIC.
3058
Implementation details. In the first stage of the training process, we use the Adam optimizer with learning rate of le-4 and train for 200 epochs, which will be reduced to le-5 and train for an additional 100 epochs for the second stage. Moreover, all models are trained with a batch size of 128, implemented in PyTorch with CUDA support, and trained on single NVIDIA A8000-80G GPU. They are all trained four times with different Lagrange parameters, obtaining four distinct compression levels regarding bpp.
Figure 4. Perceptual quality comparisons in terms of PSNR among the proposed PSIC, the LIC backbone network, and BAvAFT.
Facial attribute analysis: We utilize 10,000 images from the CelebA dataset (Liu et al., 2018), each la- beled with 40 attributes (e.g., gender, age, expression). Performance is evaluated using Top-1 accuracy.
4.2. Experimental Results
Encryption efficiency. The encryption efficiency compar- isons for image-text retrieval, image classification, and fa- cial attribute analysis are shown in Fig. 3 (a)-(e). Encourag- ing results have been observed, showing that the encrypted version proposed by PSIC can significantly mislead the tar- get CLIP model, while the full version effectively preserves the full semantic information. In particular, compared to the backbone LIC network, our encrypted version achieves an average ASR of 80.8%, 72.3%, 67. 67.0%, 51.5% for text-to- image, image-to-tert retrieval, image classification, facial attribute analysis, respectively. Meanwhile, the full ver- sion maintains comparable machine analytics performance to the AE-yyperrr Considering both versions are re- constructed from the same bitstream and exhibit mutually exclusive semantic representations, these results are remark- able, owing to our multistage trigger injection and adaptive multi-objective optimization strategies.
Image captioning: The entire validation set of Flickr8k dataset (Hodosh et al., 2013) is employed. The perfor- mance is indexed by BLEU@4, CIDEr, and SPICE. In particular, BLEU@n n analyzes the proportion of n- grams in the candidate translation that appear in the reference translation. CIDEr primarily assesses the quality of image captions by calculating the similarity between the generated description and set of refer- ence descriptions. SPICE, on the other hand, focuses on semantic accuracy by using scene graphs to compare objects, attributes, and relationships in the captions.
Human perception: The Kodak dataset (Kodak, 1993) is utilized to evaluate perceptual quality, with PSNR serving as the assessment metric.
To intuitively demonstrate the effectiveness of misleading downstream tasks, we additionally adopt the Attack Success Rate (ASR) metric for image-text retrieval, image classifica- tion, and facial attribute analysis. In particular, the ASR is defined as the percentage of samples that misclassified due to adversarial attacks. In our context, ASR measures the proportion of samples from the baseline Learned Image Compression (LIC) model that are correctly processed in downstream tasks but are misled when processed through our proposed model.
Meanwhile, compared to the cutting-edge backdoor attack method BAvAFT, the proposed PSIC offers overwhelming advantages in encryption (attacking) efficiency, achieving an average ASR improvement of 12.9% regarding all of the downstream tasks. Additionally, our full version consis- tently outperforms the clean version across multiple tasks and bpp points. According to our analysis, the main ad- vancement lies in injecting the triggers at the representation stage, rather than the preprocessing stage, as this approach allows the optimized encoder to better capture the key in- formation critical for both attacking and normal purposes. Moreover, the performance on the unseen image caption- ing task in Fig. 3 (f), (g), and (h) further demonstrates the effectiveness and superiority of our method over BAvAFT, highlighting the robustness and generalization capability of the PSIC.
Baseline, Regarding the backbone LIC network, the mile- stone -E-Hyyprprrror (Ballé et al., 2018) is adopted. Specifi- cally, both the proposed PSIC and the backbone network are trained by randomly selected 70,000 image-text pairs from the CC3M dataset (Sharma et al., 2018) from scratch for a fair comparison. Morcover, a cutting-edge LIC-oriented backdoor attack method, BAvAFT, also employed for
Perceptual Quality. Perceptual quality comparisons in
7

---

Privacy-Shiclded Image Compression: Defending Against Exploitation from Vision-Language Pretrained Models
Original Image
AF-Foyeerior Tmage
Imange (Pall)
Curr Image (Pacryplac)
BAVAFT Taage (Clen)
(Posco:
0.pp 29.69dB
0.p0/byp. 29.86JD
0.10bbpg. 28.17dB
0.133ppp. 29.17dB
0.458tpp, 27.18
LA man playing saxophone to fire hydrunt..
AE Hyperprior
plays saxophone yellow fire hydrant.
2.A man plays saxophone next to yellow tire hydrant.
Ours(Full)
3.A man stands by wall playing saxophone.
BAvAFT (Clean Poison)
4. 4aMan playing musical instrument standing beside yellow hydrant
One child pulls another, , sitting red wagon along the sand
Ours (Enerypted)
S. man standling next to fire hydrant and playing saxophone
Figure S. Visualization of performance in terms of perceptual quality and encryption efficiency.
Bill lie 12t
RBI fari2
4.3. Ablation Study
0.4
In this subsection, we perform an ablation study on the UAEO optimization function to demonstrate its effective- ness. In this ablation, we use a naive attacking objective function-i.e., minimizing the cosine similarity between the reconstructed encrypted version and its corresponding text-as the ablated version. Comparison results between our full version and the ablated version at the same compres- sion level are provided in Fig. 6. As shown, the uncertainty- aware strategy significantly boosts encryption efficiency, resulting in an average ASR improvement 6.3% and 6.2% for text-to-image and image-to-text retrieval tasks, respec- tively, thereby illustrating its effectiveness in capturing and leveraging uncertainty within the training data.
0.3
Da dlaichopp
Da Mecaaa
0.2
013
0.1
0.5
835
135
8.45
(a)
(b)
ASR De 12) 12
02-
DaniDo
Gus Eao
0.5
8.35
2.55
8.8
5. Conncluiions
0pgl
(c)
(d)
This paper proposes a novel PSIC scheme aimed at pro- tecting user privacy from exploitation by VLP models at the image compression stage. PSIC is characterized by its ability produce bitstreams with two optional decoding versions: encrypted version that offers satisfactory per- ceptual quality yet remains inaccessible to VLP models, and a full version providing complete semantic information. These capabilities are achieved through two key components. First, a CLTG module generates bias information based on customizable conditions, guiding the decoder toward differ- ent reconstructed outputs. Second, an UAEO optimization function leverages the target VLP model's uncertainty the training data to provide an efficient and robust optimization strategy. Moreover, an adaptive mujti-obeeectvv optimization approach is explored to simultaneously enhance encryption performance and perceptual quality within a unified training process. Experimental results and ablation studies validate the effectiveness our design.
Figure 6. Ablation results for the proposed UABO optimization function, illustrating encryption efficiency in (a), (b), and (c), and perceptual quality in (d).
terms of PSNR is provided in Fig. 4. As shown, our full version maintains the same rate-perception level as the back- bone LIC, while also showing an improvement over the clean mode of BAvAFT, indicated by an average increase of 1.0 dB across the four bpp points. Moreover, consider- ing that BAvAFT requires separate encoding processes for clean and poison modes, the improvement is remarkable. Meanwhile, our encrypled version also maintains favorable perceptual quality, demonstrating its capacity to meet com- mon perception-orient oriinted requirements. To provide insight into the overall performance, a group of visualized examples shown in Fig. 5.
8

---

Privacy-Shiclded Image Compression: Defending Against Exploitation from Vision-Languaae Pretrained Models
Impact Statement
prior. In Proceedings of the International Conference on Learning Representations, 2018.
As deep learning models increasingly rely on publicly avail- able visual data, ensuring ativacy-preeeviig mechanisms becomes crucial for preventing unintended data exploitation. This paper introduces privacy-shielding image compres- sion (PSIC) framework, aiming to protect image data from unauthorized use by Vision-Language Pretrained (VLP) models and general machine vision analytics. By intro- ducing multiple decoding options, it enables individuals and organizations to maintain control over their visual data while still benefiting from advanced AI technologies. Ethically, this approach promotes data ownership and responsible AI usage, reducing the risk of unauthorized surveillance unintended data exploitation. However, similar techniques could also be leveraged to potentially negative applications, including hindering lawful model training. We believe that our work contributes to ongoing discussions about ethical Al and responsible data usage, highlighting the importance of developing privacy preseeiigg techniques that balance protection with ethical considerations. This, in turn, should inspire further research toward more robust, fair, and trust- worthy Al-driven data protection strategies.
Barni, M., Kallas, K., and Tondi, B. new backdoor attack in cnns by training set corruption without label poison- ing. In 2019 IEEE International Conference on Image Processing (ICIP), pp. 101-105,2011 2019.
Chen, X., Fang, H., Lin, T.-Y.. Vedantam, R.. Gupta, S., Dollár, P., and Zitnick, C. L. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv: 504.00325, 2015.
Chen, Y., Liu, C., Huang, W., Cheng, S., Arcucci, R., and Xiong, Z. Generative text-guided 3d vision-language pretraining for unified medical image segmentation. arXiv preprint arXiv:2306.00 04811.00..1...... 2023.
Dai, W., Li, J., Li, D., Tiong, A., Zhao, J., Wang, W., Li, B., Fung, P., and Hoi, S. Instructblip: Towards general- purpose vision- languala models with instruction tuning. arxiv 2023. arXiv preprint S.XXiv2330506650 2023.
Dempster, A. Upper and lower probabilities induced by a multivalued mapping. In Classic works of the Dempster- Shafer theory of belief functions, pp. 57-72. Springer, 2008
Acknowledgements
This work was supported by the Guangdong Basic and Applied Basic Research Foundation under Grant 2024A1515010454.
Deng, J., Dong, W., Socher, R., Li, L..J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248 2018-2552220 255, 2009.
References
Du, Y., Liu, Z., Li, J., and Zhao, W. X. A survey of language pre-trained models. arXiv preprint arXiv:2202.1095. 2022.
Abdelfattah, R., Guo, Q., Li, X., Wang, X., and Wang, S. Cdul: Clip-driven unsupervised learning for multi-label image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1348- 1357, 2023.
Fang, H., Xiong. P., Xu, L., and Chen, Y. Clip2video: Mas- tering video-text retrieval via image clip. arXiv preprint arXiv:2106. 2ww:2106.11099 11097, 2021.
Antol, S., Agrawal, A., Lu, J., Mitchell, M.. Batra, D., Zitnick, C. L.., and Parikh, D. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision, pp. 2425-2433 2015.
Fu, J., Xu, S., Liu, H., Liu, Y., Xie, N., Wang, C.-C., Liu, J., Sun, Y., and Wang, B. Cma-clip: Cross-modality attention clip for text-image classification. In Proceedings of the IEEE international conference on image processing, pp. 2846-2850, 2022.
Bai, J., Gao, K., Min, S., Xia, S.-T., Li, Z., and Liu, W. Badelip: Trigger-aware prompt learning for backdoor attacks clip. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 24239-24250, 2024.
Gao, K., Bai, J., Wu, B., Ya, M., and Xia, S.-T. Impercepti- ble and robust backdoor attack in 3d point cloud. IEEE Transactions on Information Forensics and Security, pp. 1267-1282, 2023.
Ballé, J., Laparra, V., and Simoncelli, E. P. End-to-end optimized image compression. In Proceedings of the International Conference on Learning Representations, 2017.
Gu, T., Dolan-Gavite B., and Garg, S. Badnets: Identify- ing vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv: 100: 067333330......00000000777333333 2017.
Ballé, J.. Minnen, D., Singh, S., Hwang, S. J., and Johnston, N. Variational image compression with a scale hyper-
Guo, Z., Zhang, Z., Feng, R., and Chen, Z. Causal con- textual prediction for learned image compression. IEEE

---

Privacy-Shiclded Image Compression: Defending Against Exploitation from Vision-Language Pretrained Models
Transactions on Circuits and Systems for Video Technol- 0gy, pp. 2329-2341, 2021.
Liang, S., Zhu, M., Liu, A., Wu, B., Cao. X., and Chang, E.-C. Badclip: Dual-embedding guided backdoor attack multimodal contrastive learning, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24645-24657 2024.
He, D., Zheng, Y., Sun, B., Wang, Y., and Qin, H. Checker- board context model for efficient learned image compres- sion. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 14771- 14780, 2021.
Liu, Z., Luo, P., Wang, X., and Tang, X. Large-scale celeb- faces attributes (celeba) dataset. Retrieved August, pp. 11, 2018.
Hodosh, M., Young, P., and Hockenmaier. J. Framing image description as ranking task: Data, models and evaluation metrics. Journal of Artificial Intelligence Research, pp. 853-899, 2013.
Luo, H.,, Ji, L.., Zhong, M., Chen, Y., Lei, W., Duan, N., and Li, T. Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning. Neurocomputting pp. 293-304, 2022.
Jia, C., Yang, Y., Xia, Y., Chen, Y-TT., Parekh, Z., Pham, H.,, Le, Q., Sung, Y.-H., Li, Z., and Duerig, T. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, pp. 4904-4916, 2021.
Ma, Y., Xu, G., Sun, X., Yan, M., Zhang, and Ji, R. X-clip: End-to-end multi-grained contrastive learning for video- text retrieval. In Proceedings of the ACM International Conference on Multimedia, pp. 638-647, 2022.
Jiang, W., Yang. J., Zhai, Y., Ning, P., Gao, F., and Wang, R. Mlic: Multi-reference entropy model for learned image compression. In Proceedings of the ACM International Conference on Multimedia, pp. 7618 0008--77772000 2023.
Minnen, D. and Singh, S. Channel- autoregressive en- tropy models for learned image compression. In 2020 IEEE International Conference on Image Processing (ICIP), pp. 3339-334444 3343. IEEE, 2020.
Johan Modin, J. C. Contrastive language-image foren- sic search. clifs, 2022.
Peng, F., Yang, X., Xiao, L., Wang, Y., and Xu, C. Sgva-clip: Semantic- -onttidc-gudeddddee visual adapting of vision-language mod- els for few-shot image classification. IEEE Transactions on Multimedia, 2023a.
Kingma, D. and Welling, M. Auto-encoding variational bayes. arxiv: 131261 14. In Proceedings of the Interna- tional Conference on Learning Representations, 2013.
Peng, Z., Wang. W., Dong. L., Hao, Y., Huang. S., Ma, S., and Wei, F. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:23066 14824, 2023b.
Kodak, E. Kodak lossless true color image suite (photocd pcd0992). URL htp://rrk. us/graphics/kodak, pp. 2, 1993.
Qin, Y., Peng, D., Peng, X., Wang, X., and Hu, P. Deep evidential learning with noisy correspondence for cross- modal retrieval. In Proceedings of the 30th ACM Interna- tional Conference on Multimedia, pp. 4948 4956, 2022.
Li, J., Li, D., Xiong, C., and Hoi, S. Blip: Bootstrapping language-image pre-training for unified languaee understanding and generation. In International confer- on machine learning, pp. 12888-12900 12900, 2022.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G.,
Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In Proceedings of the International conference machine learning, pp. 8748 8763, 2021.
J., Li, D., Savarese, S., and Hoi. S. 2lip-2: Bootstrapping language-iman image pre-training with frozen image encoders and large language models. In International conference on muchine learning, pp. 19730-19742, 2023.
Rippel, O. and Bourdev, L. Real-time adaptive image com- pression. In Proceedings of the International Conference on Machine Learning, pp. 2922-2930, 2017.
Li, L. H., Yatskar, M. Yin, D., Hsich, C.-J., and Chang, K.-W. Visualbert: A simple and performant baseline for vision and language. arXiv preprint 2019.
Sharma, P., Ding, N., Goodman, S., and Soricut, R. Con- ceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Anmual Meeting of the Association for Compu- tational Linguistics (Volume I: Long Papers), pp. 2556 2565, 2018
Li, Y., Li, Y., Wu, Li, He, R., and Lyu. S. Invisible backdoor attack with sample-ppecific triggers. In Pro- ceedings of the IEEE/CVF international conference on computer vision, pp. (6463-16472 2021.
10

---

Privacy-Shiclded Image Compression: Defending Against Exploitation from Vision-Languaa Pretrained Models
Xu, M., Zhang, Z.,, Wei, F,, Lin, Y., Cao, Y., Hu, H., and Bai, X. A simple baseline for open-vocabulary semantic segmentation with pre-trained vision-language model. In European Conference on Computer Vision, pp. 736-753, 2022.
Yan, S., Dong, N., Zhang, I.., and Tang, J. Clip-driven fine- fine-grained text-image person re-identification. IEEE Transactions Image Processing, 2023.
Yang. H., Zhao, Y., and Qin, B. Face-sensitive image-to- emotional-text cross-modal translation for multimodal sspect-baseed sentiment analysis. In Proceedings of the 2022 conference on empirical methods in natural lan- guage processing, pp. 3324-3335, 2022.
Yang, R. and Mandt, S. Lossy image compression with conditional diffusion models. Advances in Neural Infor- mation P Processing Systems, 2024.
Yang. Z., He, X., Z., Backes, M., Humbert, M., Berrang, P. and Zhang, Y. Data poisoning attacks against multi- modal encoders. In International Conference on Machine Learning, pp. 39299-39311 2023.
Yu, Y., Wang. Y., Yang, W., Lu, S., Tan, Y.P.., and Kot, A. C. Backdoor attacks against deep image compression via adaptive frequency trigger. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 12250-12259, 2023.
Yu, Y.. Wang, Y., Yang. W., Guo. L., Lu, S., Duan, L.- Y., Tan, Y.P.,, and Kot, A. C. Robust and transferable backdoor attacks against deep image compression with selective frequency prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Zhou, H., Huang, S., Zhang, F,, and Xu, C. Ceprompt: Cross-modal emotion-aware prompting for facial expres- sion recognition. IEEE Transactions on Circuits and Systems for Video Technology, 2024.


Reviews:
[{"review_id": "34DsgK5lGI", "reviewer": "Reviewer_ZrT6", "summary": "This paper proposes Privacy-Shielded Image Compression (PSIC), a learned image compression framework that prevents Vision-Language Pretrained models from extracting semantic information while preserving perceptual quality. PSIC enables a single bitstream to be decoded into an encrypted version or a full version conditioned on user input. The method introduces a Conditional Latent Trigger Generation (CLTG) module and an Uncertainty-Aware Encryption-Oriented (UAEO) optimization function to enhance flexibility and attack effectiveness. Experimental results show that PSIC effectively disrupts VLP-based tasks, such as image-text retrieval and classification, while maintaining comparable compression quality to baseline models under the full version.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First of all, we would like to express our sincere gratitude for your responsible and constructive comments, which have been very helpful in further improving the quality of our manuscript. Below, we summarize your concerns into five key points and address them one by one. We hope our responses satisfactorily address all of your concerns. If there is any misunderstanding on our part, we would greatly appreciate any further clarification or additional suggestions.\n\nQ.1: Ablation study on the CLTG module.\n\nAns.: We provide an ablation study on CLTG. We remove all CLTG modules and train only the compression backbone using UAEO-based loss function:$λ_1 L_2(x, \\widehat{x}) + λ_2 L_{UC} (\\widehat{x}) + r$, $\\widehat{x}$ is the reconstructed image, $r$ is the bitrate. For a fair comparison, we adjusted the values of $\\lambda_1$ and $\\lambda_2$ so that the ablated version achieves the same ASR level as the full version. Due to the time constraint, we only provide comparison results at 0.20 BPP points shown below, and we will complete the ablation study in our revised paper.\n\n|Bpp|Meth.|PSNR|i2t|ASR(i2t)|t2i|ASR(t2i)|\n|-|-|-|-|-|-|-|\n|0.20|Ours($x_f$)|26.07|0.32|-|0.20|-|\n||Ours($x_e$)|25.61|0.13|0.69|0.07|0.78|\n||*w/o* CLTG|24.48|0.11|0.73|0.08|0.75|\n\nAs shown, the CLTG module’s effectiveness is evident: it significantly boosts the perceptual quality (1.59dB PSNR) at the same ASR level.\n\nQ.2: Ablation study on the multi-stage training strategy.\n\nAns.: We follow the nice suggestion to look into the effect of our multi-stage training strategy. In particular, we omit Stage 2 and instead train the model using the settings from Stage 1 until convergence (an additional 100 epochs). We present the comparison results regarding encryption performance (encrypted version) and perceptual quality (full version) below.\n\n|Bpp|Meth.|ASR(i2t)|ASR(t2i)|PSNR($x_f$/$x_e$)|\n|-|-|-|-|-|\n|0.13|Ours|0.66|0.74|24.72/24.57|\n||*w/o* stg2|0.05|0.06|24.70/24.68|\n|0.20|Ours|0.69|0.78|26.07/25.61|\n||*w/o* stg2|0.03|0.05|26.02/26.02|\n|0.30|Ours|0.74|0.83|27.37/25.67|\n||*w/o* stg2|0.04|0.06|27.28/27.27|\n|0.46|Ours|0.82|0.88|28.78/27.34|\n||*w/o* stg2|0.02|0.04|28.84/28.84|\n\nAs we stated in our manuscript, Stage 2 emphasizes enhancing the capacity for divergent representation. Without this stage, encryption performance would decrease by about 72% on average in terms of ASR.\n\nQ.3 More effective compression baselines.\n\nAns.: Thanks for your constructive comments. We have implemented the PSIC scheme in two cutting-edge LIC models (ELIC, and MLIC++). Due to time constraints, we are currently providing comparison results at a single BPP point (see below). All training configurations were kept strictly consistent with those in our manuscript to ensure fair comparisons. We will include comprehensive results on additional LIC baselines (including TIC) across multiple BPP points in the supplementary material of our revised paper. We appreciate your understanding.\n\n||Meth.|PSNR|i2t|t2i|ASR(i2t)|ASR(t2i)|\n|-|-|-|-|-|-|-|\n|ELIC(bpp=0.19)|Baseline|27.24|0.32|0.22|-|-|\n||Ours($x_e$)|26.64|0.30|0.19|-|-|\n||Ours($x_f$)|26.05|0.12|0.07|0.74|0.79|\n|MLIC++(bpp=0.18)|Baseline|27.26|0.35|0.22|-|-|\n||Ours($x_e$)|26.93|0.32|0.21|-|-|\n||Ours($x_f$)|26.44|0.13|0.09|0.69|0.73|\n\nAs shown, the proposed PSIC scheme can be readily integrated into different LIC models, demonstrating strong effectiveness in both encryption and compression performance.\n\nQ.4 Applicability to other VLP models.\n\nAns. : Following your insightful suggestion, we have adopted another milestone VLP model, ALIGN (“Scaling up visual and vision-language representation learning with noisy text supervision”, ICML2021), as the target. Specifically, we restructured the UAEO optimization function based on ALIGN, and trained the entire framework at a single BPP point (bpp = 0.13). The corresponding results are presented below, demonstrating the strong applicability of the proposed PSIC method to other VLP models.\n\n|Meth.|PSNR|i2t|ASR(i2t)|t2i|ASR(t2i)|\n|-|-|-|-|-|-|\n|Comp. Baseline|24.83|0.26|-|0.17|-|\n|$x_f$(CLIP)|24.72|0.24|-|0.17|-|\n|$x_e$(CLIP)|24.57|0.11|0.66|0.07|0.74|\n|$x_f$(ALIGN)|24.68|0.23|-|0.17|-|\n|$x_e$(ALIGN)|24.32|0.17|0.43|0.14|0.47|\nQ.5 Proofreading for typos.\n\nThank you for your thorough review. We will definitely incorporate your suggestions into our revised manuscript, and we will carefully proofread the manuscript to eliminate any typos or grammatical issues.", "claims_and_evidence": "The effectiveness of the proposed methods requires more ablation studies, such as the Conditional Latent Trigger Generation Module and the Adaptive Multi-Objective Optimization Strategy.", "methods_and_evaluation_criteria": "Yes, I believe that introducing uncertainty to constrain image-text matching is a reasonable approach and can more effectively attack VLP (Vision-Language Pretraining) models.", "theoretical_claims": "No theoretical proof.", "experimental_designs_or_analyses": "1.\tI recommend the authors to add experiments on more effective baselines like ELIC[1], TIC[2].\n2.\tThe authors should add ablations on the proposed Conditional Latent Trigger Generation Module and Adaptive Multi-Objective Optimization Strategy. In this paper, the authors only provide ablation results on proposed UAEO optimization.\n\n[1] He D, Yang Z, Peng W, et al. Elic: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 5718-5727.\n[2] M. Lu, P. Guo, H. Shi, C. Cao and Z. Ma, \"Transformer-based Image Compression,\" 2022 Data Compression Conference (DCC), Snowbird, UT, USA, 2022, pp. 469-469.", "relation_to_broader_scientific_literature": "The author considers incorporating backdoor attacks into neural image compression for VLP models, which has good practical applicability for user privacy protection on social media.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Please refer to above comments.", "other_comments_or_suggestions": "Typos:\n1.\tLine 099, \"pravicy\" → privacy.\n2.\tLine 244, \"paramterized\" → parameterized\n3.\tLine 246, \"of for the image-text pair\" → of the image-text pair\n4.\tLine 310, unmatched number of parentheses in the equation S 1-1.", "questions_for_authors": "If the VLP model is not CLIP, is the proposed method still applicable?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "No"}}, {"review_id": "eDWjfLaof6", "reviewer": "Reviewer_uJHB", "summary": "The paper presents a novel approach for privacy protection in image compression, termed Privacy-Shielded Image Compression (PSIC), aimed at defending against exploitation by Vision-Language Pretrained (VLP) models. The method leverages a flexible compression scheme that creates bitstreams with multiple decoding options. By default, the bitstream preserves perceptual quality while concealing semantic content from VLP models. The method can be adjusted to allow reconstruction of images with full semantic information when required. The system incorporates a Conditional Latent Trigger Generation (CLTG) module to produce bias information for guiding the decoding process and an Uncertainty-Aware Encryption-Oriented (UAEO) optimization function to maximize encryption performance while maintaining perceptual quality. The paper claims that PSIC can mislead VLP models and prevent them from exploiting compressed images for downstream tasks such as image-text retrieval and classification, while preserving image quality.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First of all, we deeply appreciate your kind suggestions and positive feedback on our work, which have greatly encouraged us. We hope our responses below adequately address your concerns.\n\nQ.1: Discuss the challenges of extending to other data types.\n\nAns.: Thanks for your insightful comments. Based on our experience, we believe the proposed PSIC scheme can be readily extended to different data types, especially video. In particular, end-to-end video compression follows a frame-by-frame pipeline, with a framework similar to image compression. Hence, the CLTG module, UAEO function, and associated optimization strategy can be directly integrated.\n\nMeanwhile, certain adjustments are indeed necessary to handle domain-specific structures (e.g., temporal dependencies). Specifically, in the video compression process, the bitstream for a given frame must be reconstructed and used as a contextual prior for compressing the subsequent frame, thereby removing temporal redundancy. Taking both compression and encryption needs into consideration, we recommend decoding the bitstream into the full version, as it retains more similarity to the next frame.\n\nQ.2: Discuss the UAEO’s contribution against more sophisticated attacks.\n\nAns.: Thanks for your suggestions. The UAEO potentially enhances the robustness of the PSIC method against sophisticated attacks, e.g., model inversion, by explicitly leveraging the uncertainty within VLP models and incorporating this uncertainty into corresponding constraints or loss functions. Specifically, UAEO employs Dempster-Shafer Theory to identify image-text pairs with high uncertainty (low confidence), guiding the embedding of targeted yet nearly invisible adversarial patterns during the image compression process. This approach potentially complicates an attacker's ability to reconstruct sensitive semantic information, thereby offering stronger privacy protection. \n\nAdditionally, as UAEO primarily focuses on mechanisms related to representation-level information, further complementary defense strategies could potentially be incorporated to strengthen the overall security guarantees of PSIC. Potential enhancements include adopting differential privacy techniques (e.g., “Gaussian differential privacy”, JRSS2022) to introduce controlled data obfuscation, employing multi-modal encryption methods for broader protection against cross-modal inference threats (e.g., “BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP”, CVPR2024), and regularly conducting security audits combined with adversarial retraining, which may help continuously maintain or further enhance robustness over time.\n\nQ.3: Discuss the trade-offs between encryption efficiency and compression performance.\n\nAns.: Thanks for your comments. To illustrate the trade-offs between encryption and compression performance, we present how these metrics evolve during the second training stage, where the encoder is frozen (i.e., the resulting bitstreams remain unchanged). Specifically, we obtained checkpoints (bpp=0.13) at different epochs, and evaluated their encryption efficiency and compression performance. All test settings are identical to those described in our manuscript.\n|Epoch|10|20|30|40|\n|-|-|-|-|-|\n|PSNR|24.82|24.73|24.65|24.63|\n|ASR(i2t)|0.11|0.14|0.25|0.30|\n|ASR(t2i)|0.15|0.18|0.26|0.32|\n\nAs you insightfully noted, there are indeed trade-offs between these two inherently conflicting objectives: increasing encryption efficiency comes at the expense of compression performance. Nevertheless, with the help of our CLTG module, PSIC achieves a favorable balance—maintaining compression performance comparable to standard codecs while providing significantly improved encryption strength.\n\nQ.4: Compare PSIC with JPEG or WebP.\n\nAns.: Thanks for your suggestions. The comparison results on the Kodak testing set are shown below. For a fair comparison of file size (indicated by bpp), we adjusted the quantization parameters of JPEG and WebP to achieve the same PSNR (25.85 dB) as our PSIC. Our PSIC runs on an NVIDIA 3090 GPU, while JPEG and WebP are run on an Intel Xeon Silver 4310 CPU.\n|Method|BPP|Avg Encoding / Avg Decoding (ms)|\n|-|-|-|\n|PSIC (full version)|0.13|246/30|\n|JPEG|0.28|150/2|\n|WebP|0.11|50/4|\n\nQ.5 Discuss the CLTG module’s generalization capacity.\n\nAns.: Thanks for your insightful suggestions. First, we would like to point out that the generalization performance of the CLTG module is CLOSELY TIED to the target DOWNSTREAM mode. To ensure strong generalization, we adopt the CLIP model as the target, as it has been trained on a large-scale dataset (4 billion text-image pairs). As a result, the proposed PSIC method exhibits promising generalization ability across a variety of tasks. In our experiments, the test sets used for image classification, facial attribute analysis, and image captioning were entirely UNSEEN during training, yet our method achieved strong performance in both compression and encryption metrics.", "claims_and_evidence": "Yes, clear and convincing.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Correct.", "experimental_designs_or_analyses": "The experiments cover a range of downstream tasks (e.g., image classification, image-text retrieval, and facial attribute analysis) and provide solid evidence that the PSIC method outperforms existing approaches. The use of attack success rate (ASR) to measure the model's effectiveness in misleading VLP models and its ability to retain perceptual quality is appropriate. The ablation studies further validate the contributions of specific modules, such as the UAEO function.", "relation_to_broader_scientific_literature": "This paper addresses the growing privacy concerns regarding VLP models and their ability to exploit publicly available visual data. Given the increasing reliance on machine learning models that use large-scale datasets for training, the proposed method provides a timely and important contribution to privacy-preserving data compression.", "essential_references_not_discussed": "N/A.", "other_strengths_and_weaknesses": "The approach is innovative, particularly in the integration of image compression with privacy-preserving mechanisms. The ability to generate multiple versions of an image from the same bitstream—one that protects privacy by preventing interpretation by VLP models, and one that retains full semantic information for legitimate use—is a novel idea. The use of Conditional Latent Trigger Generation and Uncertainty-Aware Encryption-Oriented optimization adds unique elements to the field of privacy-enhancing image compression.", "other_comments_or_suggestions": "N/A.", "questions_for_authors": "1. What specific steps would be needed to extend the PSIC method to other data types, such as audio or video, and how do you foresee the challenges related to temporal or sequential data?\n\n2. How does the UAEO optimization function contribute to the robustness of the PSIC method against more sophisticated attacks, such as model inversion attacks? Could further defense mechanisms be incorporated to strengthen the security guarantees?\n\n3. Could you elaborate on the trade-offs between encryption efficiency and compression performance? In particular, how does the PSIC method compare with other state-of-the-art image compression methods, such as JPEG or WebP, in terms of file size and compression time?\n\n4. Can you provide a deeper discussion on the potential limitations of the CLTG module, particularly in cases where the image content varies significantly from the training data? How might the system adapt to these situations?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "S4JjTX2FKU", "reviewer": "Reviewer_BN9M", "summary": "This paper proposes a novel Privacy-Shielded Image Compression (PSIC) method aimed at protecting images from being exploited by Vision-Language Pretrained (VLP) models. The PSIC framework integrates an adaptive multi-objective optimization strategy that balances perceptual quality and encryption effectiveness. A Conditional Latent Trigger Generation (CLTG) module is introduced to generate different decoding options from a single bitstream, while the Uncertainty-Aware Encryption-Oriented (UAEO) optimization function is used to maximize encryption efficiency against VLP models. Experimental results indicate that PSIC significantly degrades the interpretability of compressed images by VLP models while maintaining visual quality for human perception.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First of all, we greatly appreciate your thorough review and helpful suggestions. Below, we address your concerns one by one, and we hope our responses fully clarify each point. If any misunderstanding remains, we sincerely welcome further clarification or suggestions.\n\nQ.1 Incorporate more quality metrics.\n\nAns.: Thanks for your suggestion. We have further evaluated the perceptual quality using additional metrics, including SSIM, LPIPS, NIQE, and CLIPIQA. The results corresponding to Fig. 4 of our manuscript can be found at the following link. https://pic1.imgdb.cn/item/67eb75140ba3d5a1d7e8f36d.png It should be noted that the NIQE curves appear somewhat irregular and produce unusual results compared to the other four metrics. Given that NIQE is a no-reference IQA method with known instability in certain scenarios,  it may not accurately reflect the actual perceptual quality in this case.\n\nQ.2: Perform the PSIC scheme on other LIC models.\n\nAns.: We appreciate your insightful suggestion. We have implemented the PSIC in two cutting-edge LIC models (ELIC, and MLIC++). To ensure a fair comparison, all training settings strictly follow those described in our original manuscript. The corresponding results can be found in below table. \n\n||Meth.|PSNR|i2t|t2i|ASR(i2t)|ASR(t2i)|\n|-|-|-|-|-|-|-|\n|ELIC(bpp=0.19)|Baseline|27.24|0.32|0.22|-|-|\n||Ours($x_e$)|26.64|0.30|0.19|-|-|\n||Ours($x_f$)|26.05|0.12|0.07|0.74|0.79|\n|MLIC++(bpp=0.18)|Baseline|27.26|0.35|0.22|-|-|\n||Ours($x_e$)|26.93|0.32|0.21|-|-|\n||Ours($x_f$)|26.44|0.13|0.09|0.69|0.73|\n\nThese results demonstrate the effectiveness of the PSIC method on other LIC models, as it achieves perceptual quality comparable to the baseline model while offering remarkable encryption efficiency in terms of ASR.  Due to time constraints, we will include more comprehensive results on additional LIC models across multiple BPP points in the supplementary material of our revised paper. We appreciate your understanding.\n\nQ.3 Lacking citation of BAvAFT and compare with recent works (Yu et al. 2023, 2024).\n\nAns.: We apologize for the confusion caused by the missing citation for BAvAFT. In fact, BAvAFT originates from the papers you mentioned (Yu et al., 2023, 2024. The 2024 version is an extension of the 2023 version, and we employed the latest version in our manuscript). We will properly cite both works in the revised version to avoid confusion and to give appropriate confirmation.\n\nQ.4. Ablation study on CLTG module.\n\nAns.: We provide an ablation study on CLTG. We remove all CLTG modules and train only the compression backbone using UAEO-based loss function:$λ_1 L_2(x, \\widehat{x}) + λ_2 L_{UC} (\\widehat{x}) + r$, $\\widehat{x}$ is the reconstructed image, $r$ is the bitrate. We performed the comparison at a BPP level of 0.20, and the results are provided below.\n\n|Bpp|Meth.|PSNR|i2t|ASR(i2t)|t2i|ASR(t2i)|\n|-|-|-|-|-|-|-|\n|0.20|Ours($x_f$)|26.07|0.32|-|0.20|-|\n||Ours($x_e$)|25.61|0.13|0.69|0.07|0.78|\n||*w/o* CLTG|24.48|0.11|0.73|0.08|0.75|\n\nAs shown, removing CLTG leads to a PSNR drop of 1.59 dB at the same ASR level.\n\nQ.5. Ablation study on multi-stage training strategy.\n\nAns.: We further conduct an additional ablation study on the proposed multi-stage training strategy. In particular, we omit Stage 2 and instead train the model using the settings from Stage 1 until convergence (an additional 100 epochs). We present the comparison results regarding encryption performance (encrypted version) and perceptual quality (full version) below.\n\n|Bpp|Meth.|ASR(i2t)|ASR(t2i)|PSNR($x_f$/$x_e$)|\n|-|-|-|-|-|\n|0.13|Ours|0.66|0.74|24.72/24.57|\n||*w/o* stg2|0.05|0.06|24.70/24.68|\n|0.20|Ours|0.69|0.78|26.07/25.61|\n||*w/o* stg2|0.03|0.05|26.02/26.02|\n|0.30|Ours|0.74|0.83|27.37/25.67|\n||*w/o* stg2|0.04|0.06|27.28/27.27|\n|0.46|Ours|0.82|0.88|28.78/27.34|\n||*w/o* stg2|0.02|0.04|28.84/28.84|\n\nThe effectiveness of the multi-stage training strategy is readily apparent, as it improves encryption performance by about 72% in terms of the average ASR.\n\nQ.6: Provide more visual comparisons\n\nAns.: Thanks for your kind suggestions. We have provided four extra groups of visual compression in addition to Fig. 5, which can be observed in the following links. https://pic1.imgdb.cn/item/67eb9c9c0ba3d5a1d7e9030c.jpg\n\nQ.7 Provide a supplemental material.\n\nAns.: We appreciate this constructive suggestion and will incorporate the following in the supplementary material:\n\na) Extended ablation studies on the CLTG module and the multi-stage training strategy across multiple BPP points; \n\nb) Comprehensive evaluations demonstrating the implementation of PSIC on other baseline LIC models, along with multiple perceptual quality metrics; \n\nc) Additional visual examples generated by the PSIC method, the compression baseline, and the employed BAvAFT method in the supplementary material; \n\nd) A detailed introduction to the Dempster-Shafer Theory, along with the derivation process of the employed evidence extractor.", "claims_and_evidence": "The claim: \"The proposed PSIC scheme is plug-and-play and can be seamlessly integrated into most existing learned image compression (LIC) models\" is not well supported. The paper only evaluated PSIC on one backbone (AE-Hyperprior, 2018). How about more recent LIC models such as HiFiC[1], ELIC[2] and MLIC[3]? If the experiments on these more recent models are not possibly all provided, please clarify what cost it will take to equip the general LIC model with the proposed PSIC method, to support that \"the proposed PSIC scheme is plug-and-play.\" \n\n[1] High-fidelity generative image compression[C]. Advances in neural information processing systems, 2020, 33: 11913-11924.\n\n[2] ELIC: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 5718-5727.\n\n[3] MLIC: Multi-reference entropy model for learned image compression[C]//Proceedings of the 31st ACM International Conference on Multimedia. 2023: 7618-7627.", "methods_and_evaluation_criteria": "The method is compared against BAvAFT, but the baseline method lacks proper citation. A clear description of BAvAFT should be provided.", "theoretical_claims": "The Dempster-Shafer Theory used to model pair uncertainty is presented as a black-box implementation with minimal discussion. More discussion would be helpful to understand.", "experimental_designs_or_analyses": "1. The ablation study in the experiment part is useful but would be better to extend to analyze other proposed modules or strategies, such as the proposed adaptive multi-objective optimization.\n2. In perceptual quality comparison, instead of comparing to AE-Hyperprior (2018), more recent general LIC models that optimized for perceptual quality should be included to provide baselines. Additionally, more quality metrics should be evaluated, such as those that are more aligned with the human vision system: SSIM, LPIPS, NIQE, CLIPIQA.\n3. The paper evaluates the method on four downstream tasks: image-text retrieval, image classification, facial attribute analysis, and image captioning. These are reasonable benchmarks, but additional comparison with recent works in Yu et al. 2023, 2024 (which is reviewed in related work) is necessary, since these works are also designed for misleading downstream machine analytic tasks.", "relation_to_broader_scientific_literature": "Previous related works are either focusing on misleading downstream machine analytic tasks or designing pre-processing methods on the input data. The paper focuses on privacy shielding against exploitation from vision-language pretrained models, and it addresses this problem from the aspect of the compression phase, which is interesting and novel.", "essential_references_not_discussed": "BAvAFT mentioned in the experiment section is not cited, and it is unclear if it is a fair baseline.", "other_strengths_and_weaknesses": "Strengths: \n1. The proposed Privacy-Shielded Image Compression (PSIC) framework introduces a novel approach to integrating privacy protection directly into the image compression process, which is an underexplored area in learned image compression.\n2. The idea of conditional latent trigger generation (CLTG) to enable different decoding versions from a single bitstream is innovative and provides flexibility in balancing privacy and perceptual quality.\n3. The method is designed to be plug-and-play, potentially making it applicable to a broad range of learned image compression (LIC) models.\n\nWeaknesses:\n\nMy identified weaknesses have been thoroughly mentioned in other parts of the review, so no additional weaknesses will be mentioned here.", "other_comments_or_suggestions": "In addition to the one row of images in Fig. 5, please show more visual comparison results to demonstrate the effectiveness of the proposed method (both full and encrypted modes) on perceptual quality.", "questions_for_authors": "1. Why does the experimental backbone use AE-Hyperprior (2018)? Have you considered evaluating PSIC on newer LIC models?\n2. Why is there no direct comparison with Yu et al. (2023, 2024), given that their work also targets misleading downstream machine analysis tasks?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents Privacy-Shielded Image Compression (PSIC), a learned image compression framework designed to preserve perceptual quality while protecting semantic privacy from Vision-Language Pretrained (VLP) models. PSIC enables a single bitstream to be decoded into an encrypted version or a full version conditioned on user input.  Key components include the Conditional Latent Trigger Generation (CLTG) module and the Uncertainty-Aware Encryption-Oriented (UAEO) optimization, which together aim to mislead VLP models while maintaining compression quality.
The paper introduces a promising and practical direction for privacy-preserving image compression, especially in the context of rising concerns over VLP model misuse. While additional experiments and refinements would strengthen the work, the core idea and demonstrated potential justify acceptance.
Decision: Accept (poster)