Original: A CTIVE I MAGE I NDEXING
Revised: A CTIVE I MAGE I NDEXING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Image copy detection and retrieval from large databases leverage two components.
Revised: Image copy detection and retrieval from large databases leverage two components.

Original: First, a neural network maps an image to a vector representation, that is relatively robust to various transformations of the image.
Revised: First, a neural network maps an image to a vector representation, that is relatively robust to various transformations of the image.

Original: Second, an efficient but approximate similarity search algorithm trades scalability (size and speed) against quality of the search, thereby introducing a source of error.
Revised: Second, an efficient but approximate similarity search algorithm trades scalability (size and speed) against quality of the search, thereby introducing a source of error.

Original: This paper improves the robustness of image copy detection with active indexing , that optimizes the interplay of these two components.
Revised: This paper improves the robustness of image copy detection with active indexing , that optimizes the interplay of these two components.

Original: We reduce the quantization loss of a given image representation by making imperceptible changes to the image before its release.
Revised: We reduce the quantization loss of a given image representation by making imperceptible changes to the image before its release.

Original: The loss is back-propagated through the deep neural network back to the image, under perceptual constraints.
Revised: The loss is back-propagated through the deep neural network back to the image, under perceptual constraints.

Original: These modifications make the image more retrievable.
Revised: These modifications make the image more retrievable.

Original: Our experiments show that the retrieval and copy detection of activated images is significantly improved.
Revised: Our experiments show that the retrieval and copy detection of activated images is significantly improved.

Original: For instance, activation improves by +40% the Recall 1@1 on various image transformations, and for several popular indexing structures based on product quantization and locality sensitivity hashing.
Revised: For instance, activation improves by +40% the Recall1@1 on various image transformations, and for several popular indexing structures based on product quantization and locality sensitivity hashing.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: The traceability of images on a media sharing platform is a challenge: they are widely used, easily edited and disseminated both inside and outside the platform.
Revised: The traceability of images on a media sharing platform is a challenge: they are widely used, easily edited and disseminated both inside and outside the platform.

Original: In this paper, we tackle the corresponding task of Image Copy Detection (ICD), i.e . finding whether an image already exists in the database; and if so, give back its identifier.
Revised: In this paper, we tackle the corresponding task of Image Copy Detection (ICD), i.e . finding whether an image already exists in the database; and if so, give back its identifier.

Original: ICD methods power reverse search engines, photography service providers checking copyrights, or media platforms moderating and tracking down malicious content ( e.g .
Revised: ICD methods power reverse search engines, photography service providers checking copyrights, or media platforms moderating and tracking down malicious content ( e.g .

Original: Microsoft’s PhotoDNA (2009) or Apple’s NeuralHash (2021)).
Revised: Microsoft’s PhotoDNA (2009) or Apple’s NeuralHash (2021)).

Original: State-of-the-art image identification systems have to be robust to identify images that are edited  after their release (Douze et al., 2021;
Revised: Image identification systems have to be robust to identify images that are edited (cropping, colorimetric change, JPEG compression ...) after their release (Douze et al., 2021;

Original: Wang et al., 2022).
Revised: Wang et al., 2022).

Original: For instance, a crop, a colorimetric filter and a JPEG compression may be applied.

Original: The common approach for content-based image retrieval reduces images to high-dimensional vectors, referred to as representations .
Revised: The common approach for content-based image retrieval reduces images to high-dimensional vectors, referred to as representations .

Original: Early representations used for retrieval were hand-crafted fea-
Revised: Early representations used for retrieval were hand-crafted features such as color histograms (Swain & Ballard, 1991), GIST (Oliva & Torralba, 2001), or Fisher

Original: [Figure]
Revised: Index

Revised: Outside world

Original: Image editing
Revised: Image editing

Original: [Figure]
Revised: Original image

Original: Publication I* t(I*)
Revised: Publication I* t(I*)

Original: I o
Revised: I o

Original: Indexing
Revised: Indexing

Original: Search results
Revised: Search results

Original: Activated image
Revised: Activated image

Original: Activationf( I o ) f(t( I * )) f( I * ) f(t( I o ))
Revised: Activation

Original: Latent 
Revised: Latent space

Original: [Figure] Approximate 
Revised: Approximate Search f

Original: [Figure]
Revised: ( I o ) f(t( I * ))

Original: tures such as color histograms (Swain & Ballard, 1991), GIST (Oliva & Torralba, 2001), or Fisher
Revised: f( I * ) f(t( I o ))

Original: Vectors (Perronnin et al., 2010).
Revised: Vectors (Perronnin et al., 2010).

Original: As of now, a large body of work on self-supervised learning focuses on producing discriminative representations with deep neural networks, which has inspired recent
Revised: As of now, a large body of work on self-supervised learning focuses on producing discriminative representations with deep neural networks, which has inspired recent

Original: ICD systems.
Revised: ICD systems.

Original: In fact, all submissions to the NeurIPS2021 Image Similarity challenge (Papakipos et al., 2022) exploit neural networks.
Revised: In fact, all submissions to the NeurIPS2021 Image Similarity challenge (Papakipos et al., 2022) exploit neural networks.

Original: They are trained to provide invariance to potential image transformations, akin to data augmentation in self-supervised learning.
Revised: They are trained to provide invariance to potential image transformations, akin to data augmentation in self-supervised learning.

Original: Scalability is another key requirement of image similarity search: searching must be fast on largescale databases, which exhaustive vector comparisons cannot do.
Revised: Scalability is another key requirement of image similarity search: searching must be fast on largescale databases, which exhaustive vector comparisons cannot do.

Original: In practice, ICD engines leverage approximate neighbor search algorithms, that trade search accuracy against scalability.
Revised: In practice, ICD engines leverage approximate neighbor search algorithms, that trade search accuracy against scalability.

Original: Approximate similarity search algorithms speed up the search by not computing the exact distance between all representations in the dataset (Johnson et al., 2019; Guo et al., 2020).
Revised: Approximate similarity search algorithms speed up the search by not computing the exact distance between all representations in the dataset (Johnson et al., 2019; Guo et al., 2020).

Original: First they lower the number of scored items by partitioning the representation space, and evaluate the distances of only a few subsets.
Revised: First they lower the number of scored items by partitioning the representation space, and evaluate the distances of only a few subsets.

Original: Second, they reduce the computational cost of similarity evaluation with quantization or binarization.
Revised: Second, they reduce the computational cost of similarity evaluation with quantization or binarization.

Original: These mechanisms make indexing methods subject to the curse of dimensionality.
Revised: These mechanisms make indexing methods subject to the curse of dimensionality.

Original: In particular, in high-dimensional spaces, vector representations lie close to boundaries of the partition (B¨ohm et al., 2001).
Revised: In particular, in high-dimensional spaces, vector representations lie close to boundaries of the partition (B¨ohm et al., 2001).

Original: Since edited versions of an original image have noisy vector representations, they sometimes fall into different subsets or are not quantized the same way by the index.
Revised: Since edited versions of an original image have noisy vector representations, they sometimes fall into different subsets or are not quantized the same way by the index.

Original: All in all, it makes approximate similarity search very sensitive to perturbations of the edited image representations, which causes images to evade detection.
Revised: All in all, it makes approximate similarity search very sensitive to perturbations of the edited image representations, which causes images to evade detection.

Original: In this paper, we introduce a method that improves similarity search on large databases, provided that the platform or photo provider can modify the images before their release (see Fig. 1).
Revised: In this paper, we introduce a method that improves similarity search on large databases, provided that the platform or photo provider can modify the images before their release (see Fig. 1).

Original: We put the popular saying “attack is the best form of defense” into practice by  drawing inspiration from adversarial attacks and applying image perturbations ourselves.
Revised: We put the popular saying “attack is the best form of defense” into practice by applying image perturbations and drawing inspiration from adversarial attacks.

Original: Indeed, representations produced with neural networks are subject to adversarial examples (Szegedy et al., 2013): small perturbations of the input image can lead to very different vector representations, making it possible to create adversarial queries that fool image retrieval systems (Liu et al., 2019;
Revised: Indeed, representations produced with neural networks are subject to adversarial examples (Szegedy et al., 2013): small perturbations of the input image can lead to very different vector representations, making it possible to create adversarial queries that fool image retrieval systems (Liu et al., 2019;

Original: Tolias et al., 2019; Dolhansky & Ferrer, 2020).
Revised: Tolias et al., 2019; Dolhansky & Ferrer, 2020).

Original: In contrast, we modify an image to make it more indexing friendly.
Revised: In contrast, we modify an image to make it more indexing friendly.

Original: With minimal changes in the image domain, the image representation is pushed towards the center of the indexing partition, rising the odds that edited versions will remain in the same subset.
Revised: With minimal changes in the image domain, the image representation is pushed towards the center of the indexing partition, rising the odds that edited versions will remain in the same subset.

Original: This property is obtained by minimizing an indexation loss by gradient descent back to the image pixels, like for adversarial examples.
Revised: This property is obtained by minimizing an indexation loss by gradient descent back to the image pixels, like for adversarial examples.

Original: For indexing structures based on  quantization (Jegou et al., 2010), this strategy amounts to pushing the representation closer to its quantized codeword, in which case the indexation loss is simply measured by the reconstruction error.
Revised: For indexing structures based on product quantization (Jegou et al., 2010), this strategy amounts to pushing the representation closer to its quantized codeword, in which case the indexation loss is simply measured by the reconstruction error.

Original: Since the image quality is an important constraint here, the perturbation is shaped by perceptual filters to remain invisible to the human eye.
Revised: Since the image quality is an important constraint here, the perturbation is shaped by perceptual filters to remain invisible to the human eye.

Original: Our contributions are:
Revised: Our contributions are:

Original: • a new approach to improve ICD, when images can be changed before release; • an adversarial image optimization scheme that adds minimal perceptual perturbations to images in order to reduce reconstruction errors, and improve vector representation for indexing; • experimental evidence that the method significantly improves index performance.
Revised: • a new approach to improve ICD and retrieval, when images can be changed before release; • an adversarial image optimization scheme that adds minimal perceptual perturbations to images in order to reduce reconstruction errors, and improve vector representation for indexing; • experimental evidence that the method significantly improves index performance.

Original:  P RELIMINARIES : R EPRESENTATION L EARNING AND I NDEXING
Revised:  P RELIMINARIES : R EPRESENTATION L EARNING AND I NDEXING

Original: For the sake of simplicity, the exposure focuses onthe image representation SSCD  (Pizzi et al., 2022) and the indexing technique IVF-PQ (Jegou et al., 2010), since both are typically used for ICD.
Revised: For the sake of simplicity, the exposure focuses on image representations from SSCD networks (Pizzi et al., 2022) and the indexing technique IVF-PQ (Jegou et al., 2010), since both are typically used for ICD.

Original: Extensions to other methods can be found in Sec. 5.4.
Revised: Extensions to other methods can be found in Sec. 5.4.

Original: 2. D EEP DESCRIPTOR LEARNING
Revised: 2. D EEP DESCRIPTOR LEARNING

Original: Metric embedding learning aims to learn a mapping f : R c × h × w → R d , such that measuring the similarity between images I and I ′ amounts to computing the distance ∥
Revised: Metric embedding learning aims to learn a mapping f : R c × h × w → R d , such that measuring the similarity between images I and I ′ amounts to computing the distance ∥

Original: f ( I ) − f
Revised: f ( I ) − f

Original: ( I ′ )
Revised: ( I ′ )

Original: ∥ .
Revised: ∥ .

Original: In recent works, f is typically a neural network trained with self-supervision on raw data to learn metrically meaningful representations.
Revised: In recent works, f is typically a neural network trained with self-supervision on raw data to learn metrically meaningful representations.

Original: Methods include contrastive learning (Chen et al., 2020), self-distillation (Grill et al., 2020; Caron et al., 2021), or masking random patches of images (He et al., 2022; Assran et al., 2022).
Revised: Methods include contrastive learning (Chen et al., 2020), selfdistillation (Grill et al., 2020; Caron et al., 2021), or masking random patches of images (He et al., 2022; Assran et al., 2022).

Original: In particular, SSCD (Pizzi et al., 2022) is a training method specialized for ICD.
Revised: In particular, SSCD (Pizzi et al., 2022) is a training method specialized for ICD.

Original: It employs the contrastive self-supervision method SimCLR (Chen et al., 2020) and entropy regularization (Sablayrolles et al., 2019) to improve the separation.
Revised: It employs the contrastive self-supervised method SimCLR (Chen et al., 2020) and entropy regularization (Sablayrolles et al., 2019) to improve the distribution of the representations.

Original: 2. I NDEXING
Revised: 2. I NDEXING

Original: Given a dataset X = { x i } ni =1 ⊂ R d of d -dimensional vector representations extracted from n images and a query vector x q , wefirst consider the indexing task that addresses the problem:
Revised: Given a dataset X = { x i } ni =1 ⊂ R d of d -dimensional vector representations extracted from n images and a query vector x q , we consider the indexing task that addresses the problem:

Original: [Equation]
Revised: [Equation]

Original: This exact nearest neighbor search is not tractable over large-scale databases.
Revised: This exact nearest neighbor search is not tractable over large-scale databases.

Original: Approximate search algorithms lower the amount of scored items thanks to space partitioning and/or accelerate the computations of distances thanks to quantization and pre-computation.
Revised: Approximate search algorithms lower the amount of scored items thanks to space partitioning and/or accelerate the computations of distances thanks to quantization and pre-computation.

Original: Space partitioning and cell-probe algorithms.
Revised: Space partitioning and cell-probe algorithms.

Original: As a first approximation, nearest neighbors are sought only within a fraction of X : at indexing time, X is partitioned into X =
Revised: As a first approximation, nearest neighbors are sought only within a fraction of X : at indexing time, X is partitioned into X =

Original: (cid:83) bi =1 X i .
Revised: (cid:83) bi =1 X i .

Original: At search time, an algorithm Q : R d → { 1 , .., b } k ′ determines a subset of k ′ buckets in which to search, such that k ′ =
Revised: At search time, an algorithm Q : R d → { 1 , .., b } k ′ determines a subset of k ′ buckets in which to search, such that k ′ =

Original: | Q ( x q ) | ≪ b , yielding the approximation:
Revised: | Q ( x q ) | ≪ b , yielding the approximation:

Original: [Equation]
Revised: [Equation]

Original: A well known partition is the KD-tree (Bentley, 1975) that divides the space along predetermined directions.
Revised: A well known partition is the KD-tree (Bentley, 1975) that divides the space along predetermined directions.

Original: Subsequently, locality sensitive hashing (LSH) (Indyk & Motwani, 1998; Gionis et al., 1999) and derivative (Datar et al., 2004;
Revised: Subsequently, locality sensitive hashing (LSH) (Indyk & Motwani, 1998; Gionis et al., 1999) and derivative (Datar et al., 2004;

Original: Paulev ´ e et al., 2010) employ various hash functions for bucket assignment, which implicitly partitions the space.
Revised: Paulev ´ e et al., 2010) employ various hash functions for bucket assignment, which implicitly partitions the space.

Original: We focus on the popular clustering and Inverted Files methods (Sivic & Zisserman, 2003), herein denoted by IVF.
Revised: We focus on the popular clustering and Inverted Files methods (Sivic & Zisserman, 2003), herein denoted by IVF.

Original: They employ a codebook C
Revised: They employ a codebook C

Original: = { c i } ki =1 ⊂ R d of k centroids (also called “visual words” in a local descriptor context), for instance learned with k-means over a training set of representations.
Revised: = { c i } ki =1 ⊂ R d of k centroids (also called “visual words” in a local descriptor context), for instance learned with k-means over a training set of representations.

Original: Then, Q associates x to its nearest centroid q c ( x ) such that the induced partition is the set of the k Vorono ¨ ı cells.
Revised: Then, Q associates x to its nearest centroid q c ( x ) such that the induced partition is the set of the k Vorono ¨ ı cells.

Original: When indexing x , the IVF stores x in the bucket associated with c i = q c ( x ) .
Revised: When indexing x , the IVF stores x in the bucket associated with c i = q c ( x ) .

Original: When querying x q , IVF searches only the k ′ buckets associated to centroids
Revised: When querying x q , IVF searches only the k ′ buckets associated to centroids

Original: c i nearest to x q .
Revised: c i nearest to x q .

Original: Efficient metric computation and product quantization.
Revised: Efficient metric computation and product quantization.

Original: Another approximation comes from compressed-domain distance estimation.
Revised: Another approximation comes from compressed-domain distance estimation.

Original: Vector Quantization (VQ) maps a representation x ∈ R d to a codeword q f ( x ) ∈ C
Revised: Vector Quantization (VQ) maps a representation x ∈ R d to a codeword q f ( x ) ∈ C

Original: = { C i } Ki =1 .
Revised: = { C i } Ki =1 .

Original: The function q f is often referred to a quantizer and C i as a reproduction value .
Revised: The function q f is often referred to a quantizer and C i as a reproduction value .

Original: The vector x is then stored as an integer in { 1 , .., K } corresponding to q f ( x ) .
Revised: The vector x is then stored as an integer in { 1 , .., K } corresponding to q f ( x ) .

Original: The distance between x and query x q is approximated by ∥
Revised: The distance between x and query x q is approximated by ∥

Original: q f ( x ) − x q ∥ , which is an “asymmetric” distance computation (ADC) because the query is not compressed.
Revised: q f ( x ) − x q ∥ , which is an “asymmetric” distance computation (ADC) because the query is not compressed.

Original: This leads to:
Revised: This leads to:

Original: [Equation]
Revised: [Equation]

Original: Binary quantizers (a.k.a.
Revised: Binary quantizers (a.k.a.

Original: sketches, Charikar (2002) lead to efficient computations but inaccurate distance estimates (Weiss et al., 2008).
Revised: sketches, Charikar (2002) lead to efficient computations but inaccurate distance estimates (Weiss et al., 2008).

Original: Product Quantization (PQ) (Jegou et al., 2010) or derivatives Ge
Revised: Product Quantization (PQ) (Jegou et al., 2010) or derivatives Ge

Original: et al.
Revised: et al.

Original: (2013) offer better estimates.
Revised: (2013) offer better estimates.

Original: In PQ, a vector x ∈ R d is split into m subvectors in R d/m : x = ( x 1 , . . .
Revised: In PQ, a vector x ∈ R d is split into m subvectors in R d/m : x = ( x 1 , . . .

Original: , x m ) .
Revised: , x m ) .

Original: The product quantizer then quantizes the subvectors:
Revised: The product quantizer then quantizes the subvectors:

Original: q f : x (cid:55)→ ( q 1 ( x 1 ) , . . .
Revised: q f : x (cid:55)→ ( q 1 ( x 1 ) , . . .

Original: , q m ( x m )) .
Revised: , q m ( x m )) .

Original: If each subquantizer q j has K s reproduction values, the resulting quantizer q f has a high K =
Revised: If each subquantizer q j has K s reproduction values, the resulting quantizer q f has a high K =

Original: ( K s ) m .
Revised: ( K s ) m .

Original: The squared distance estimate is decomposed as:
Revised: The squared distance estimate is decomposed as:

Original: [Equation]
Revised: [Equation]

Original: This is efficient since x is stored by the index as q f ( x ) which has m log 2 K s bits, and since summands can be precomputed without requiring decompression at search time.
Revised: This is efficient since x is stored by the index as q f ( x ) which has m log 2 K s bits, and since summands can be precomputed without requiring decompression at search time.

Original:  A CTIVE I NDEXING
Revised:  A CTIVE I NDEXING

Original: Active indexing takes as input an image I o , adds the image representation to the index and outputs an activated image I ⋆ with better traceability properties for the index.
Revised: Active indexing takes as input an image I o , adds the image representation to the index and outputs an activated image I ⋆ with better traceability properties for the index.

Original: It makes the feature representation produced by the neural network more compliant with the indexing structure.
Revised: It makes the feature representation produced by the neural network more compliant with the indexing structure.

Original: The activated image is the one that is disseminated on the platform, therefore the alteration must not degrade the perceived quality of the image.
Revised: The activated image is the one that is disseminated on the platform, therefore the alteration must not degrade the perceived quality of the image.

Original: Images are activated by an optimization on their pixels.
Revised: Images are activated by an optimization on their pixels.

Original: The general optimization problem reads:
Revised: The general optimization problem reads:

Original: [Equation]
Revised: [Equation]

Original: ) where L is an indexation loss dependent on the indexing structure, C
Revised: ) where L is an indexation loss dependent on the indexing structure, C

Original: ( I o ) is the set of images perceptually close to I o .
Revised: ( I o ) is the set of images perceptually close to I o .

Original: Algorithm 1 and Figure 1 provide an overview of active indexing.
Revised: Algorithm 1 and Figure 1 provide an overview of active indexing.

Original: 3. I MAGE OPTIMIZATION DEDICATED TO IVF-PQ (“ ACTIVATION ”)
Revised: 3. I MAGE OPTIMIZATION DEDICATED TO IVF-PQ (“ ACTIVATION ”)

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: The indexing structure IVF-PQ involves a coarse quantizer q c built with k-means clustering for space partitioning, and a fine product quantizer q f on the residual vectors, such that a vector x ∈ R d is approximated by q ( x )
Revised: The indexing structure IVF-PQ involves a coarse quantizer q c built with k-means clustering for space partitioning, and a fine product quantizer q f on the residual vectors, such that a vector x ∈ R d is approximated by q ( x )

Original: =
Revised: =

Original: q c ( x ) + q f ( x − q c ( x )) .
Revised: q c ( x ) + q f ( x − q c ( x )) .

Original: We solve the optimization problem (5) by iterative gradient descent, back-propagating through the neural network back to the image.
Revised: We solve the optimization problem (5) by iterative gradient descent, back-propagating through the neural network back to the image.

Original: The method is classically used in adversarial example generation (Szegedy et al., 2013; Carlini & Wagner, 2017) and watermarking (Vukoti ´ c et al., 2020; Fernandez et al., 2022).
Revised: The method is classically used in adversarial example generation (Szegedy et al., 2013; Carlini & Wagner, 2017) and watermarking (Vukoti ´ c et al., 2020; Fernandez et al., 2022).

Original: Given an original image I o , the loss is an aggregation of the following objectives:
Revised: Given an original image I o , the loss is an aggregation of the following objectives:

Original: [Equation]
Revised: [Equation]

Original: with
Revised: with

Original: [Equation]
Revised: [Equation]

Original: L i is a regularization on the image distortion.
Revised: L i is a regularization on the image distortion.

Original: L f is the indexation loss that operates on the representation space.
Revised: L f is the indexation loss that operates on the representation space.

Original: L f is the Euclidean distance between x and the target q ( x o ) and its goal is to push the image feature towards q ( x o ) .
Revised: L f is the Euclidean distance between x and the target q ( x o ) and its goal is to push the image feature towards q ( x o ) .

Original: With IVF-PQ as index, the representation of the activated image gets closer to the quantized version of the original representation, but also closer to the coarse centroid.
Revised: With IVF-PQ as index, the representation of the activated image gets closer to the quantized version of the original representation, but also closer to the coarse centroid.

Original: Finally, the losses are combined as L ( I ; I o )
Revised: Finally, the losses are combined as L ( I ; I o )

Original: =
Revised: =

Original: L f ( x, q ( x o ))
Revised: L f ( x, q ( x o ))

Original: + λ L i ( I, I o ) .
Revised: + λ L i ( I, I o ) .

Original: 3. P ERCEPTUAL ATTENUATION
Revised: 3. P ERCEPTUAL ATTENUATION

Original: It is common to optimize a perturbation δ added to the image, rather than the image itself.
Revised: It is common to optimize a perturbation δ added to the image, rather than the image itself.

Original: The adversarial example literature often considers perceptual constraints in the form of an ℓ p -norm bound applied on δ (Madry et al.
Revised: The adversarial example literature often considers perceptual constraints in the form of an ℓ p -norm bound applied on δ (Madry et al.

Original: (2018) use ∥ δ ∥ ∞ < ε = 8 / 255 ).
Revised: (2018) use ∥ δ ∥ ∞ < ε = 8 / 255 ).

Original: Although a smaller ε makes the perturbation less visible, this constraint is not optimal for the human visual system (HVS), e.g . perturbations are more noticeable on flat than on textured areas of the image (see App. A.2).
Revised: Although a smaller ε makes the perturbation less visible, this constraint is not optimal for the human visual system (HVS), e.g . perturbations are more noticeable on flat than on textured areas of the image (see App. A.2).

Original: We employ a handcrafted perceptual attenuation model based on a Just Noticeable Difference (JND) map (Wu et al., 2017), that adjusts the perturbation intensity according to luminance and contrast masking.
Revised: We employ a handcrafted perceptual attenuation model based on a Just Noticeable Difference (JND) map (Wu et al., 2017), that adjusts the perturbation intensity according to luminance and contrast masking.

Original: Given an image I , the JND map H JND ( I ) ∈
Revised: Given an image I , the JND map H JND ( I ) ∈

Original: R c × h × w models the minimum difference perceivable by the HVS at each pixel and additionally rescales the perturbation channel-wise since the human eye is more sensible to red and green than blue color shift (see App.
Revised: R c × h × w models the minimum difference perceivable by the HVS at each pixel and additionally rescales the perturbation channel-wise since the human eye is more sensible to red and green than blue color shift (see App.

Original: A for details).
Revised: A for details).

Original: The relation that links the image I sent to f , δ being optimized and the original I o , reads:
Revised: The relation that links the image I sent to f , δ being optimized and the original I o , reads:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: with α a global scaling parameter that controls the strength of the perturbation and ⊙ the pointwise multiplication.
Revised: with α a global scaling parameter that controls the strength of the perturbation and ⊙ the pointwise multiplication.

Original: Coupled with the regularization L i (6), it enforces that the activated image is perceptually similar, i.e .
Revised: Coupled with the regularization L i (6), it enforces that the activated image is perceptually similar, i.e .

Original: I ⋆ ∈ C
Revised: I ⋆ ∈ C

Original: ( I o ) as required in (5).
Revised: ( I o ) as required in (5).

Original: 3. I MPACT ON THE INDEXING PERFORMANCE
Revised: 3. I MPACT ON THE INDEXING PERFORMANCE

Original: Figure 1 illustrates that the representation of the activated image gets closer to the reproduction value q ( f ( I o )) , and farther away from the Vorono¨ı boundary.
Revised: Figure 1 illustrates that the representation of the activated image gets closer to the reproduction value q ( f ( I o )) , and farther away from the Vorono¨ı boundary.

Original: This is expected to make image similarity search more robust because (1) it decreases the probability that x = f ( t ( I o )) “falls” outside the bucket; and (2) it lowers the distance between x and q ( x ) , improving the PQ distance estimate.
Revised: This is expected to make image similarity search more robust because (1) it decreases the probability that x = f ( t ( I o )) “falls” outside the bucket; and (2) it lowers the distance between x and q ( x ) , improving the PQ distance estimate.

Original: Besides, by design, the representation stored by the index is invariant to the activation.
Revised: Besides, by design, the representation stored by the index is invariant to the activation.

Original: Formally stated, consider two images I , J , and one activated version J ⋆ together with their representations x, y, y ⋆ .
Revised: Formally stated, consider two images I , J , and one activated version J ⋆ together with their representations x, y, y ⋆ .

Original: When querying x = f ( I ) , the distance estimate is ∥ q ( y ⋆ ) − x
Revised: When querying x = f ( I ) , the distance estimate is ∥ q ( y ⋆ ) − x

Original: ∥ =
Revised: ∥ =

Original: ∥ q ( y ) − x ∥ , so the index is oblivious to the change J → J ⋆ .
Revised: ∥ q ( y ) − x ∥ , so the index is oblivious to the change J → J ⋆ .

Original: This means that the structure can index passive and activated images at the same time.
Revised: This means that the structure can index passive and activated images at the same time.

Original: Retrieval of activated images is more accurate but the performance on passive images does not change.
Revised: Retrieval of activated images is more accurate but the performance on passive images does not change.

Original: This compatibility property makes it possible to select only a subset of images to activate, but also to activate already-indexed images at any time.
Revised: This compatibility property makes it possible to select only a subset of images to activate, but also to activate already-indexed images at any time.

Original:  A NALYSES
Revised:  A NALYSES

Original: We provide insights on the method for IVF-PQ, considering the effects of quantization and space partitioning.
Revised: We provide insights on the method for IVF-PQ, considering the effects of quantization and space partitioning.

Original: For an image I whose representation is x = f ( I ) ∈ R d , ˆ x denotes the representation of a transformed version: ˆ x = f ( t ( I ))
Revised: For an image I whose representation is x = f ( I ) ∈ R d , ˆ x denotes the representation of a transformed version: ˆ x = f ( t ( I ))

Original: ∈ R d , and x ⋆ the representation of the activated image I ⋆ .
Revised: ∈ R d , and x ⋆ the representation of the activated image I ⋆ .

Original: For details on the images and the implementation used in the experimental validations, see Sec. 5.1.
Revised: For details on the images and the implementation used in the experimental validations, see Sec. 5.1.

Original: 4. P RODUCT QUANTIZATION : IMPACT ON DISTANCE ESTIMATE
Revised: 4. P RODUCT QUANTIZATION : IMPACT ON DISTANCE ESTIMATE

Original: We start by analyzing the distance estimate considered by the index:
Revised: We start by analyzing the distance estimate considered by the index:

Original: [Equation]
Revised: [Equation]

Original: The activation aims to reduce the first term, i.e . the quantization error ∥ x − q ( x ) ∥ 2 , which in turn reduces ∥ ˆ x − q ( x ) ∥ 2 .
Revised: The activation aims to reduce the first term, i.e . the quantization error ∥ x − q ( x ) ∥ 2 , which in turn reduces ∥ ˆ x − q ( x ) ∥ 2 .

Original: Figure 3 shows in blue the empirical distributions of ∥ x − q ( x ) ∥ 2 (passive) and ∥ x ⋆ − q ( x ) ∥ 2 (activated).
Revised: Figure 3 shows in blue the empirical distributions of ∥ x − q ( x ) ∥ 2 (passive) and ∥ x ⋆ − q ( x ) ∥ 2 (activated).

Original: As expected the latter has a lower mean, but also a stronger variance.
Revised: As expected the latter has a lower mean, but also a stronger variance.

Original: The variation of the following factors may explain this: i) the strength of the perturbation (due to the
Revised: The variation of the following factors may explain this: i) the strength of the perturbation (due to the

Original: HVS modeled by H JND in (8)), ii) the sensitivity of the feature extractor ∥∇ x f ( x ) ∥ (some features are easier to push than others), iii) the shapes and sizes of the Vorono ¨ ı cells of PQ.
Revised: HVS modeled by H JND in (8)), ii) the sensitivity of the feature extractor ∥∇ x f ( x ) ∥ (some features are easier to push than others), iii) the shapes and sizes of the Vorono ¨ ı cells of PQ.

Original: The second term of (9) models the impact of the image transformation in the feature space.
Revised: The second term of (9) models the impact of the image transformation in the feature space.

Original: Comparing the orange and blue distributions in Fig.
Revised: Comparing the orange and blue distributions in Fig.

Original: 3, we see that it has a positive mean, but the shift is bigger for activated images.
Revised: 3, we see that it has a positive mean, but the shift is bigger for activated images.

Original: We can assume that the third term has null expectation for two reasons: i) the noise ˆ x − x is independent of q ( x ) and centered around 0, ii) in the high definition regime, quantification noise x − q ( x ) is independent of x and centered on 0.
Revised: We can assume that the third term has null expectation for two reasons: i) the noise ˆ x − x is independent of q ( x ) and centered around 0, ii) in the high definition regime, quantification noise x − q ( x ) is independent of x and centered on 0.

Original: Thus, this term only increases the variance.
Revised: Thus, this term only increases the variance.

Original: Since x ⋆ − q ( x ) has smaller norm, this increase is smaller for activated images.
Revised: Since x ⋆ − q ( x ) has smaller norm, this increase is smaller for activated images.

Original: All in all, ∥ ˆ x ⋆ − q ( x ) ∥ 2 has a lower mean but a stronger variance than its passive counterpart ∥
Revised: All in all, ∥ ˆ x ⋆ − q ( x ) ∥ 2 has a lower mean but a stronger variance than its passive counterpart ∥

Original: ˆ x − q ( x ) ∥ 2 .
Revised: ˆ x − q ( x ) ∥ 2 .

Original: Nevertheless, the decrease of the mean is so large that it compensates the larger variance.
Revised: Nevertheless, the decrease of the mean is so large that it compensates the larger variance.

Original: The orange distribution in active indexing is further away from the green distribution for negative pairs, i.e .
Revised: The orange distribution in active indexing is further away from the green distribution for negative pairs, i.e .

Original: the distance between an indexed vector q ( x ) and an independent query y .
Revised: the distance between an indexed vector q ( x ) and an independent query y .

Original: [Figure]
Revised: [Figure]

Original: 4. S PACE PARTITIONING : IMPACT ON THE IVF PROBABILITY OF FAILURE
Revised: 4. S PACE PARTITIONING : IMPACT ON THE IVF PROBABILITY OF FAILURE

Original: We denote by p f := P ( q c ( x ) ̸
Revised: We denote by p f := P ( q c ( x ) ̸

Original: = q c (ˆ x ))
Revised: = q c (ˆ x ))

Original: the probability that ˆ x is assigned to a wrong bucket by IVF assignment q c .
Revised: the probability that ˆ x is assigned to a wrong bucket by IVF assignment q c .

Original: In the single-probe search ( k ′ = 1 ), the recall (probability that a pair is detected when it is a true match, for a given threshold τ on the distance) is upper-bounded by 1 − p f :
Revised: In the single-probe search ( k ′ = 1 ), the recall (probability that a pair is detected when it is a true match, for a given threshold τ on the distance) is upper-bounded by 1 − p f :

Original: R τ = P ( { q c (ˆ x ) =
Revised: R τ = P ( { q c (ˆ x ) =

Original: q c ( x ) } ∩ {∥ ˆ x − q ( x ) ∥ < τ } ) ≤ P ( { q c (ˆ x ) =
Revised: q c ( x ) } ∩ {∥ ˆ x − q ( x ) ∥ < τ } ) ≤ P ( { q c (ˆ x ) =

Original: q c ( x ) } )
Revised: q c ( x ) } )

Original: = 1 − p f .
Revised: = 1 − p f .

Original: In other terms, even with a high threshold τ → ∞ (and low precision), the detection misses representations that ought to be matched, with probability p f .
Revised: In other terms, even with a high threshold τ → ∞ (and low precision), the detection misses representations that ought to be matched, with probability p f .

Original: It explains the sharp drop at recall R = 0 .
Revised: It explains the sharp drop at recall R = 0 .

Original: 13 in
Revised: 13 in

Original: Fig.
Revised: Fig.

Original: This is why it is crucial to decrease p f .
Revised: This is why it is crucial to decrease p f .

Original: The effect of active indexing is to reduce ∥ ˆ x − q c ( x ) ∥ therefore reducing p f and increasing the upper-bound for R : the drop shifts towards R = 0 .
Revised: The effect of active indexing is to reduce ∥ ˆ x − q c ( x ) ∥ therefore reducing p f and increasing the upper-bound for R : the drop shifts towards R = 0 .

Original: This explanation suggests that pushing x towards q c ( x ) decreases even more efficiently p f .
Revised: This explanation suggests that pushing x towards q c ( x ) decreases even more efficiently p f .

Original: This makes the IVF more robust to transformation but this may jeopardize the PQ search because features of activated images are packed altogether.
Revised: This makes the IVF more robust to transformation but this may jeopardize the PQ search because features of activated images are packed altogether.

Original: In a way, our strategy, which pushes x towards q ( x ) , dispatches the improvement over the IVF and the PQ search.
Revised: In a way, our strategy, which pushes x towards q ( x ) , dispatches the improvement over the IVF and the PQ search.

Original:  E XPERIMENTAL R ESULTS 5. E XPERIMENTAL SETUP
Revised:  E XPERIMENTAL R ESULTS 5. E XPERIMENTAL SETUP

Original: Dataset.
Revised: Dataset.

Original: We use DISC21 (Douze et al., 2021) a dataset dedicated to ICD.
Revised: We use DISC21 (Douze et al., 2021) a dataset dedicated to ICD.

Original: It includes 1M reference images and 50k query images, 10k of which are true copies from reference images.
Revised: It includes 1M reference images and 50k query images, 10k of which are true copies from reference images.

Original: A disjoint 1Mimage set with same distribution as the reference images is given for training.
Revised: A disjoint 1Mimage set with same distribution as the reference images is given for training.

Original: Images resolutions range from 200 × 200 to 1024 × 1024 pixels (most of the images are around 1024 × 768 pixels).
Revised: Images resolutions range from 200 × 200 to 1024 × 1024 pixels (most of the images are around 1024 × 768 pixels).

Original: The queries used in our experiments are not the queries in DISC21, since we need to control the image transformations in our experiments, and most transformations of DISC21 were done manually so they are not reproducible.
Revised: The queries used in our experiments are not the queries in DISC21, since we need to control the image transformations in our experiments, and most transformations of DISC21 were done manually so they are not reproducible.

Original: Our queries are transformations of images after active indexing .
Revised: Our queries are transformations of images after active indexing .

Original: These transformations range from simple attacks like rotation to more realistic social network transformations which created the original DISC21 queries (see App.
Revised: These transformations range from simple attacks like rotation to more realistic social network transformations which created the original DISC21 queries (see App.

Original: B.1).
Revised: B.1).

Original: Metrics.
Revised: Metrics.

Original: For retrieval, our main metric is Recall 1 @ 1 ( R @ 1 for simplicity), which corresponds to the proportion of positive queries where the top-1 retrieved results is the reference image.
Revised: For retrieval, our main metric is Recall 1 @ 1 ( R @ 1 for simplicity), which corresponds to the proportion of positive queries where the top-1 retrieved results is the reference image.

Original: For copy detection, we use the same metric as the NeurIPS Image Similarity Challenge (Douze et al., 2021).
Revised: For copy detection, we use the same metric as the NeurIPS Image Similarity Challenge (Douze et al., 2021).

Original: We retrieve the k = 10 most similar database features for every query; and we declare a pair is a match if the distance is lower than a threshold τ .
Revised: We retrieve the k = 10 most similar database features for every query; and we declare a pair is a match if the distance is lower than a threshold τ .

Original: To evaluate detection efficiency, we use the 10k matching queries above-mentioned together with 40k negative queries ( i.e . not included in the database).
Revised: To evaluate detection efficiency, we use the 10k matching queries above-mentioned together with 40k negative queries ( i.e . not included in the database).

Original: We use precision and recall, as well as the area under the precision-recall curve, which is equivalent to the micro average precision ( µ AP ).
Revised: We use precision and recall, as well as the area under the precision-recall curve, which is equivalent to the micro average precision ( µ AP ).

Original: While R @ 1 only measures ranking quality of the index, µ AP takes into account the confidence of a match.
Revised: While R @ 1 only measures ranking quality of the index, µ AP takes into account the confidence of a match.

Original: As for image quality metric, we use the Peak Signal-to-Noise Ratio (PSNR) which is defined as 10 log 10 (cid:0) 255 2 / MSE( I, I ′ ) 2 (cid:1) , as well as SSIM (Wang et al., 2004) and the norm ∥ I − I ′ ∥ ∞ .
Revised: As for image quality metric, we use the Peak Signal-to-Noise Ratio (PSNR) which is defined as 10 log 10 (cid:0) 255 2 / MSE( I, I ′ ) 2 (cid:1) , as well as SSIM (Wang et al., 2004) and the norm ∥ I − I ′ ∥ ∞ .

Original: Implementation details.
Revised: Implementation details.

Original: The evaluation procedure is: (1) we train an index on the 1M training images, (2) index the 1M reference images, (3) activate (or not) 10k images from this reference set.
Revised: The evaluation procedure is: (1) we train an index on the 1M training images, (2) index the 1M reference images, (3) activate (or not) 10k images from this reference set.

Original: (4) At search time, we use the index to get closest neighbors (and their distances) of transformed versions from a query set made of the 10k images.
Revised: (4) At search time, we use the index to get closest neighbors (and their distances) of transformed versions from a query set made of the 10k images.

Original: Unless stated otherwise, we use a IVF4096,PQ8x8 index (IVF quantizer with 4096 centroids, and PQ with 8 subquantizers of 2 8 centroids), and use only one probe on IVF search for shortlist selection ( k ′ = 1 ).
Revised: Unless stated otherwise, we use a IVF4096,PQ8x8 index (IVF quantizer with 4096 centroids, and PQ with 8 subquantizers of 2 8 centroids), and use only one probe on IVF search for shortlist selection ( k ′ = 1 ).

Original: Compared to a realistic setting, we voluntarily use an indexing method that severely degrades learned representations to showcase and analyze the effect of the active indexing.
Revised: Compared to a realistic setting, we voluntarily use an indexing method that severely degrades learned representations to showcase and analyze the effect of the active indexing.

Original: For feature extraction, we use an SSCD model with a ResNet50 trunk (He et al., 2016).
Revised: For feature extraction, we use an SSCD model with a ResNet50 trunk (He et al., 2016).

Original: It takes image resized to 288 × 288 and generates normalized representations in R 512 .
Revised: It takes image resized to 288 × 288 and generates normalized representations in R 512 .

Original: Optimization (5) is done with the Adam optimizer (Kingma & Ba, 2015), the learning rate is set to 1 , the number of iterations to
Revised: Optimization (5) is done with the Adam optimizer (Kingma & Ba, 2015), the learning rate is set to 1 , the number of iterations to

Original: N = 10 and the regularization to λ = 1 .
Revised: N = 10 and the regularization to λ = 1 .

Original: In (8), the distortion scaling is set to α = 3 (leading to an average PNSR around 43 dB).
Revised: In (8), the distortion scaling is set to α = 3 (leading to an average PNSR around 43 dB).

Original: In this setup, activating 128 images takes around 6s ( ≈ 40ms/image) with a 32GB GPU.
Revised: In this setup, activating 128 images takes around 6s ( ≈ 40ms/image) with a 32GB GPU.

Original: It can be sped-up at the cost of some accuracy (see App. C.2).
Revised: It can be sped-up at the cost of some accuracy (see App. C.2).

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: 5. A CTIVE VS . P ASSIVE
Revised: 5. A CTIVE VS . P ASSIVE

Original: This section compares retrieval performance of active and passive indexing.
Revised: This section compares retrieval performance of active and passive indexing.

Original: We evaluate R @1 when different transformations are applied to the 10k reference images before search.
Revised: We evaluate R @1 when different transformations are applied to the 10k reference images before search.

Original: The “Passive” lines of Tab. 1 show how the IVF-PQ degrades the recall.
Revised: The “Passive” lines of Tab. 1 show how the IVF-PQ degrades the recall.

Original: This is expected, but the IVF-PQ also accelerates search 500 × and the index is 256 × more compact, which is necessary for large-scale applications.
Revised: This is expected, but the IVF-PQ also accelerates search 500 × and the index is 256 × more compact, which is necessary for large-scale applications.

Original: Edited images are retrieved more often when they were activated for the index: increase of up to +60 R @ 1 for strong brightness and contrast changes, close to results of the brute-force search.
Revised: Edited images are retrieved more often when they were activated for the index: increase of up to +60 R @ 1 for strong brightness and contrast changes, close to results of the brute-force search.

Original: We also notice that the performance of the active IVF-PQ k ′ =1 is approximately the same as the one of the passive IVF-PQ k ′ =16 , meaning that the search can be made more efficient at equal performance.
Revised: We also notice that the performance of the active IVF-PQ k ′ =1 is approximately the same as the one of the passive IVF-PQ k ′ =16 , meaning that the search can be made more efficient at equal performance.

Original: For the IVF-PQ † that does less approximation in the search (but is slower and takes more memory), retrieval on activated images is also improved, though to a lesser extent.
Revised: For the IVF-PQ † that does less approximation in the search (but is slower and takes more memory), retrieval on activated images is also improved, though to a lesser extent.

Original: As for copy detection, Figure 2 gives the precision-recall curves obtained for a sliding value of τ , and corresponding µ AP.
Revised: As for copy detection, Figure 2 gives the precision-recall curves obtained for a sliding value of τ , and corresponding µ AP.

Original: Again, we observe a significant increase ( × 2 ) in µ AP with active indexing.
Revised: Again, we observe a significant increase ( × 2 ) in µ AP with active indexing.

Original: Note that the detection performance is much weaker than the brute-force search even in the active case because of the strong approximation made by space partitioning (more details in Sec. 4.2).
Revised: Note that the detection performance is much weaker than the brute-force search even in the active case because of the strong approximation made by space partitioning (more details in Sec. 4.2).

Original: Example of activated images are given in Fig.
Revised: Example of activated images are given in Fig.

Original: 5 (more in App. D), while the qualitative image metrics are as follows:
Revised: 5 (more in App. E), while the qualitative image metrics are as follows:

Original: PSNR = 43 .
Revised: PSNR = 43 .

Original: 8 ± 2 . 2 dB, SSIM = 0 .
Revised: 8 ± 2 . 2 dB, SSIM = 0 .

Original: 98 ± 0 . 01 , and ∥ I − I ′
Revised: 98 ± 0 . 01 , and ∥ I − I ′

Original: ∥ ∞ = 14 .
Revised: ∥ ∞ = 14 .

Original: 5 ± 1 . 2 .
Revised: 5 ± 1 . 2 .

Original: These results are computed on 10k images, the ± indicates the standard deviation.
Revised: These results are computed on 10k images, the ± indicates the standard deviation.

Original: 5. PSNR T RADE - OFF
Revised: 5. I MAGE QUALITY TRADE - OFF

Original: For a fixed index and neural extractor, the performance of active indexing mainly depends on the scaling α that controls the activated image quality.
Revised: For a fixed index and neural extractor, the performance of active indexing mainly depends on the scaling α that controls the activated image quality.

Original: In Fig.
Revised: In Fig.

Original: 4, we repeat the previous experiment for different values of α and plot the µ AP against the average PSNR.
Revised: 4, we repeat the previous experiment for different values of α and plot the µ AP against the average PSNR.

Original: As expected, lower PSNR implies better µ AP ( e.g . at PSNR 30 dB, the µ AP is augmented threefold compared to the passive case).
Revised: As expected, lower PSNR implies better µ AP. For instance, at PSNR 30 dB, the µ AP is augmented threefold compared to the passive case.

Original: Indeed, for strong perturbations the objective function of (6) can be further lowered, reducing even more the gap between representations and their quantized counterparts.
Revised: Indeed, for strong perturbations the objective function of (6) can be further lowered, reducing even more the gap between representations and their quantized counterparts.

Original: 5. G ENERALIZATION Generalizationto other neural feature extractors.
Revised: 5. G ENERALIZATION Generalization

Revised: to other neural feature extractors.

Original: We first reproduce the experiment of Sec. 5.1 with different extractors, that cover distinct training methods and architectures.
Revised: We first reproduce the experiment of Sec. 5.1 with different extractors, that cover distinct training methods and architectures.

Original: Among them, we evaluate a ResNext101 (Xie et al., 2017) trained with
Revised: Among them, we evaluate a ResNext101 (Xie et al., 2017) trained with

Original: SSCD (Pizzi et al., 2022), a larger network than the ResNet50 used in our main experiments ; the winner of the descriptor track of the NeurIPS ISC, L YAKAAP -dt

Revised: SSCD (Pizzi et al., 2022), a larger network than the ResNet50 used in our main experiments ; the winner of the descriptor track of the NeurIPS ISC, L YAKAAP -dt

Original: (Yokoo, 2021), that uses an
Revised: (Yokoo, 2021), that uses an

Original: EfficientNetv2 architecture (Tan & Le, 2021) ; networks from DINO (Caron et al., 2021), either based on ResNet50 or ViT (Dosovitskiy et al., 2021), like the ViT-S model (Touvron et al., 2021).
Revised: EfficientNetv2 architecture (Tan & Le, 2021) ; networks from DINO (Caron et al., 2021), either based on ResNet50 or ViT (Dosovitskiy et al., 2021), like the ViT-S model (Touvron et al., 2021).

Original: [Table]
Revised: [Table]

Original: Table 2 presents the R @ 1 obtained on 10k activated images when applying different transformations before search.
Revised: Table 2 presents the R @ 1 obtained on 10k activated images when applying different transformations before search.

Original: The R @ 1 is better for activated images for all transformations and all neural networks.
Revised: The R @ 1 is better for activated images for all transformations and all neural networks.

Original: The average improvement on all transformations ranges from +12% for DINO ViT-s to +30% for SSCD ResNet50.
Revised: The average improvement on all transformations ranges from +12% for DINO ViT-s to +30% for SSCD ResNet50.

Original: Generalization to other indexes.
Revised: Generalization to other indexes.

Original: The method easily generalizes to other types of indexing structures, the only difference being in the indexation loss L f (6).
Revised: The method easily generalizes to other types of indexing structures, the only difference being in the indexation loss L f (6).

Original: We present some of them below:
Revised: We present some of them below:

Original: • PQ and OPQ .
Revised: • PQ and OPQ .

Original: In PQ (Jegou et al., 2010), a vector x ∈ R d is approximated by q f ( x ) .
Revised: In PQ (Jegou et al., 2010), a vector x ∈ R d is approximated by q f ( x ) .

Original: L f reads ∥ x − q f ( x o ) ∥ .
Revised: L f reads ∥ x − q f ( x o ) ∥ .

Original: In OPQ (Ge et al., 2013), vectors are rotated by matrix R before codeword assignment, such that RR ⊤
Revised: In OPQ (Ge et al., 2013), vectors are rotated by matrix R before codeword assignment, such that RR ⊤

Original: =
Revised: =

Original: I .
Revised: I .

Original: L f becomes ∥ x − R ⊤ q f ( Rx o ) ∥ .
Revised: L f becomes ∥ x − R ⊤ q f ( Rx o ) ∥ .

Original: • IVF.
Revised: • IVF.

Original: Here, we only do space partitioning.
Revised: Here, we only do space partitioning.

Original: Employing L f = ∥ x − q c ( x o ) ∥ (“pushing towards the cluster centroid”) decreases the odds of x falling in the wrong cell (see Sec. 4.2).
Revised: Employing L f = ∥ x − q c ( x o ) ∥ (“pushing towards the cluster centroid”) decreases the odds of x falling in the wrong cell (see Sec. 4.2).

Original: In this case, an issue can be that similar representations are all pushed together to a same centroid, which makes them less discriminate.
Revised: In this case, an issue can be that similar representations are all pushed together to a same centroid, which makes them less discriminate.

Original: Empirically
Revised: Empirically

Original: we found that this does not happen because
Revised: we found that this does not happen because

Original: perceptual
Revised: perceptual

Original: constraint in the image
Revised: constraint in the image

Original: domain
Revised: domain

Original: prevents
Revised: prevents

Original: features
Revised: features

Original: from getting too close.
Revised: from getting too close.

Original: • LSH.
Revised: • LSH.

Original: Locality Sensitive Hashing maps x ∈ R d to a binary hash b ( x ) ∈ R L .
Revised: Locality Sensitive Hashing maps x ∈ R d to a binary hash b ( x ) ∈ R L .

Original: It is commonly done with projections against a set of vectors, which give for j ∈ [1 , .., L ] , b j ( x ) = sign( w ⊤ j x ) .
Revised: It is commonly done with projections against a set of vectors, which give for j ∈ [1 , .., L ] , b j ( x ) = sign( w ⊤ j x ) .

Original: The objective L f = − 1 /L
Revised: The objective L f = − 1 /L

Original: (cid:80) j sign( b ( x o )) · w ⊤ j x , allows to push x along the LSH directions and to improve the robustness of the hash.
Revised: (cid:80) j sign( b ( x o )) · w ⊤ j x , allows to push x along the LSH directions and to improve the robustness of the hash.

Original: Table 3 presents the R @ 1 and µ AP obtained on the 50k query set.
Revised: Table 3 presents the R @ 1 and µ AP obtained on the 50k query set.

Original: Again, results are always better in the active scenario.
Revised: Again, results are always better in the active scenario.

Original: We remark that active indexing has more impact on space partitioning tech-
Revised: We remark that active indexing has more impact on space partitioning tech-

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: niques: the improvement for IVF is higher than with PQ and the LSH binary sketches.
Revised: niques: the improvement for IVF is higher than with PQ and the LSH binary sketches.

Original: As to be expected, the impact is smaller when the indexing method is more accurate.
Revised: As to be expected, the impact is smaller when the indexing method is more accurate.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Image watermarking hides a message in a host image, such that it can be reliably decoded even if the host image is edited.
Revised: Image watermarking hides a message in a host image, such that it can be reliably decoded even if the host image is edited.

Original: Early methods directly embed the watermark signal in the spatial or transform domain like DCT or DWT (Cox et al., 2007).
Revised: Early methods directly embed the watermark signal in the spatial or transform domain like DCT or DWT (Cox et al., 2007).

Original: Recently, deep-learning based methods jointly train an encoder and a decoder to learn how to watermark images (Zhu et al., 2018; Ahmadi et al., 2020; Zhang et al., 2020).
Revised: Recently, deep-learning based methods jointly train an encoder and a decoder to learn how to watermark images (Zhu et al., 2018; Ahmadi et al., 2020; Zhang et al., 2020).

Original: Image watermarking is an alternative technology for ICD.
Revised: Watermarking is an alternative technology for ICD.

Original: In contrast with indexing, the image is modified before publication.Our proposal bridges indexing and watermarking.
Revised: Our method bridges indexing and watermarking, where the image is modified before publication.

Original: Our perceptual model (8) is also well suited for watermarking.
Revised: Regarding retrieval performance, active indexing is more robust than watermarking.

Original: We expect active indexing to be more robust than watermarking because the embedded signal reinforces the structure naturally present in the original image, whereas watermarking has to hide a large secret key in the signal with security guarantees.
Revised: Indeed, the embedded signal reinforces the structure naturally present in the original image, whereas watermarking has to hide a large secret keyed signal independent of the original feature. App. D provides a more thorough discussion and experimental results comparing indexing and watermarking.

Original: Active fingerprint is more related to our work.
Revised: Active fingerprint is more related to our work.

Original: As far as we know, this concept has been invented by Voloshynovskiy et al.
Revised: As far as we know, this concept was invented by Voloshynovskiy et al.

Original: They consider that the image I ∈ R N is mapped to x ∈ R N by an invertible transform W such that WW ⊤ .
Revised: They consider that the image I ∈ R N is mapped to x ∈ R N by an invertible transform W such that WW ⊤ .

Original: The binary fingerprint is obtained by taking the sign of the projections of x against a set of vectors b 1 , .
Revised: The binary fingerprint is obtained by taking the sign of the projections of x against a set of vectors b 1 , .

Original: , b L ∈ R N
Revised: , b L ∈ R N

Original: ( ` a la LSH).
Revised: (`a la LSH).

Original: Then, they change x to strengthen the amplitude of these projections so that their signs become more robust to noise.
Revised: Then, they change x to strengthen the amplitude of these projections so that their signs become more robust to noise.

Original: They recover I ⋆ with W ⊤ .
Revised: They recover I ⋆ with W ⊤ .

Original: This scheme is applied to image patches in (Kostadinov et al., 2016) where the performance is measured as a bit error rate after JPEG compression.
Revised: This scheme is applied to image patches in (Kostadinov et al., 2016) where the performance is measured as a bit error rate after JPEG compression.

Original: Our paper adapts this idea from fingerprinting to indexing, with modern deep learning representations and state-of-the-art indexing techniques.
Revised: Our paper adapts this idea from fingerprinting to indexing, with modern deep learning representations and state-of-the-art indexing techniques.

Original: The range of transformations is also much broader and includes geometric transforms.
Revised: The range of transformations is also much broader and includes geometric transforms.

Original:  C ONCLUSION & D ISCUSSION
Revised:  C ONCLUSION & D ISCUSSION

Original: We introduce a way to improve ICD in large-scale settings, when images can be changed before release.
Revised: We introduce a way to improve ICD in large-scale settings, when images can be changed before release.

Original: It leverages an optimization scheme, similar to adversarial examples, that modifies images so that (1) their representations are better suited for indexing, (2) the perturbation is invisible to the human eye.
Revised: It leverages an optimization scheme, similar to adversarial examples, that modifies images so that (1) their representations are better suited for indexing, (2) the perturbation is invisible to the human eye.

Original: We provide grounded analyses on the method and show that it significantly improves retrieval performance of activated images, on a number of neural extractors and indexing structures.
Revised: We provide grounded analyses on the method and show that it significantly improves retrieval performance of activated images, on a number of neural extractors and indexing structures.

Original: Activating images take times but one advantage is that the database may contain both active and passive images: active indexing does not spoil the performance of passive indexing and vice-versa.
Revised: Activating images takes time (in the order of 10 ms/image) but one advantage is that the database may contain both active and passive images: active indexing does not spoil the performance of passive indexing and vice-versa.

Original: This is good for legacy compliance and also opens the door to flexible digital asset management strategies (actively indexing images of particular importance).
Revised: This is good for legacy compliance and also opens the door to flexible digital asset management strategies (actively indexing images of particular importance).

Original: The method has several limitations.
Revised: The method has several limitations.

Original: First, it is not agnostic to the indexing structure and extractor that are used by the similarity search.
Revised: First, it is not agnostic to the indexing structure and extractor that are used by the similarity search.

Original: Second, an adversary could break the indexing system in several ways.
Revised: Second, an adversary could break the indexing system in several ways.

Original: In a black-box setting (no knowledge of the indexing structure and neural network extractor), adversarial purification (Shi et al., 2021) could get rid of the perturbation that activated the image.
Revised: In a black-box setting (no knowledge of the indexing structure and neural network extractor), adversarial purification (Shi et al., 2021) could get rid of the perturbation that activated the image.

Original: In a semi-white-box setting (knowledge of the feature extractor), targeted mismatch attacks against passive indexing like Tolias et al.
Revised: In a semi-white-box setting (knowledge of the feature extractor), targeted mismatch attacks against passive indexing like Tolias et al.

Original: (2019) may also work.
Revised: (2019) may also work.

Original: Adversarial training (Madry et al., 2018) could be a defense.
Revised: Adversarial training (Madry et al., 2018) could be a defense.

Original: For instance, it is interesting to know if adversarial training prevents active indexing, or if the perceptual perturbation that is used in our method is still able to push features in the latent space of a robust and defended neural network.
Revised: For instance, it is interesting to know if adversarial training prevents active indexing, or if the perceptual perturbation that is used in our method is still able to push features in the latent space of a robust and defended neural network.

Original: E THICS S TATEMENT
Revised: E THICS S TATEMENT

Original: Societal impact statement.
Revised: Societal impact statement.

Original: Content tracing is a double-edged sword.
Revised: Content tracing is a double-edged sword.

Original: On the one hand, it allows media platforms to more accurately track malicious content (pornographic, terrorist, violent images, e.g . Apple’s NeuralHash and Microsoft’s PhotoDNA) and to protect copyright ( e.g .
Revised: On the one hand, it allows media platforms to more accurately track malicious content (pornographic, terrorist, violent images, e.g . Apple’s NeuralHash and Microsoft’s PhotoDNA) and to protect copyright ( e.g .

Original: Youtube’s Content ID).
Revised: Youtube’s Content ID).

Original: On the other hand it can be used as a means of societal and political censorship, to restrict free speech of specific communities.
Revised: On the other hand it can be used as a means of societal and political censorship, to restrict free speech of specific communities.

Original: However, we still believe that research needs to be advanced to improve global moderation in the internet.
Revised: However, we still believe that research needs to be advanced to improve global moderation in the internet.

Original: We also believe that advantages that a better copy detection could bring are more numerous than its drawbacks.
Revised: We also believe that advantages that a better copy detection could bring are more numerous than its drawbacks.

Original: Environmental impact statement.
Revised: Environmental impact statement.

Original: We roughly estimated that the total GPU-days used for running all our experiments to 200 , or ≈ 5000 GPU-hours.
Revised: We roughly estimated that the total GPU-days used for running all our experiments to 200 , or ≈ 5000 GPU-hours.

Original: Experiments were conducted using a private infrastructure and we estimate total emissions to be in the order of a ton CO 2 eq.
Revised: Experiments were conducted using a private infrastructure and we estimate total emissions to be in the order of a ton CO 2 eq.

Original: Estimations were conducted using the MachineLearning Impact calculator presented in Lacoste et al.
Revised: Estimations were conducted using the MachineLearning Impact calculator presented in Lacoste et al.

Original: We do not consider in this approximation: memory storage, CPU-hours, production cost of GPUs/ CPUs, etc.
Revised: We do not consider in this approximation: memory storage, CPU-hours, production cost of GPUs/ CPUs, etc.

Original: as well as the environmental cost of training the neural networks used as feature extractors.
Revised: as well as the environmental cost of training the neural networks used as feature extractors.

Original: Although the cost of the experiments and the method is high, it could possibly allow to reduce the computations needed in large databases thanks to improved performance of indexing structures.
Revised: Although the cost of the experiments and the method is high, it could possibly allow a reduction of the computations needed in large data-centers thanks to improved performance of indexing structures.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original:  Models used for feature extraction (SSCD, DINO, ISC-dt1) can be downloaded in their respective repositories.The implementation will be made available.
Revised: The implementation will be made available. Models used for feature extraction (SSCD, DINO, ISC-dt1) can be downloaded in their respective repositories.

Original: It builds upon the open-source Pytorch Paszke et al. and FAISS Johnson et al. (2019) libraries.
Revised: It builds upon the open-source Pytorch (Paszke et al., 2019) and FAISS (Johnson et al., 2019) libraries.

Original: The main dataset used in the experiments (DISC21) can be freely downloaded on its webpage https://ai.facebook.com/datasets/disc21-dataset/. Dataset processing is described in App.
Revised: The main dataset used in the experiments (DISC21) can be freely downloaded on its webpage https://ai.facebook.com/datasets/disc21-dataset/. Dataset processing is described in App.

Original: B.1.
Revised: B.1.

Original: Supplementary material - Active Image Indexing A D ETAILS ON THE P ERCEPTUAL A TTENUATION M ODEL A. J UST N OTICEABLE D IFFERENCE MAP
Revised: Supplementary material - Active Image Indexing A D ETAILS ON THE P ERCEPTUAL A TTENUATION M ODEL A. J UST N OTICEABLE D IFFERENCE MAP

Original: The maximum change that the human visual system (HVS) cannot perceive is sometimes referred to as the just noticeable difference (JND) Krueger (1989).
Revised: The maximum change that the human visual system (HVS) cannot perceive is sometimes referred to as the just noticeable difference (JND) Krueger (1989).

Original: It is used in many applications, such as image/video watemarking, compression, quality assessment (JND is also used in audio).
Revised: It is used in many applications, such as image/video watemarking, compression, quality assessment (JND is also used in audio).

Original: JND models in pixel domain directly calculate the JND at each pixel location ( i.e . how much pixel difference is perceivable by the HVS).
Revised: JND models in pixel domain directly calculate the JND at each pixel location ( i.e . how much pixel difference is perceivable by the HVS).

Original: The JND map that we use is based on the work of Chou &
Revised: The JND map that we use is based on the work of Chou &

Original: Li (1995).
Revised: Li (1995).

Original: We use this model for its simplicity, its efficiency and its good qualitative results.
Revised: We use this model for its simplicity, its efficiency and its good qualitative results.

Original: More complex HVS models could also be used if even higher imperceptibility is needed (Watson (1993);
Revised: More complex HVS models could also be used if even higher imperceptibility is needed (Watson (1993);

Original: Yang et al.
Revised: Yang et al.

Original: ; Zhang et al.
Revised: ; Zhang et al.

Original: ; Jiang et al.
Revised: ; Jiang et al.

Original: (2022) to cite a few).
Revised: (2022) to cite a few).

Original: The JND map takes into account two characteristics of the HVS, namely the luminance adaptation (LA) and the contrast masking (CM) phenomena.
Revised: The JND map takes into account two characteristics of the HVS, namely the luminance adaptation (LA) and the contrast masking (CM) phenomena.

Original: We follow the same notations as Wu et al.
Revised: We follow the same notations as Wu et al.

Original: The CM map M C is a function of the image gradient magnitude C l (the Sobel filter of the image):
Revised: The CM map M C is a function of the image gradient magnitude C l (the Sobel filter of the image):

Original: M
Revised: M

Original: [Equation]
Revised: [Equation]

Original: 115 ×
Revised: 115 ×

Original: [Equation]
Revised: [Equation]

Original: , with
Revised: , with

Original: [Equation]
Revised: [Equation]

Original: (11) where x is the pixel location, I ( x ) the image intensity, α = 16 , and β = 26 .
Revised: (11) where x is the pixel location, I ( x ) the image intensity, α = 16 , and β = 26 .

Original: It is an increasing function of C l , meaning that the stronger the gradient is at x , the more the image is masking a local perturbation, and the higher the noticeable pixel difference is.
Revised: It is an increasing function of C l , meaning that the stronger the gradient is at x , the more the image is masking a local perturbation, and the higher the noticeable pixel difference is.

Original: LA takes into account the fact that the HVS presents different sensitivity to background luminance ( e.g .
Revised: LA takes into account the fact that the HVS presents different sensitivity to background luminance ( e.g .

Original: it is less sensible in dark backgrounds).
Revised: it is less sensible in dark backgrounds).

Original: It is modeled as:
Revised: It is modeled as:

Original: [Equation]
Revised: [Equation]

Original: where B ( x ) is the background luminance, which is calculated as the mean luminance value of a local patch centered on x .
Revised: where B ( x ) is the background luminance, which is calculated as the mean luminance value of a local patch centered on x .

Original: Finally, both effects are combined with a nonlinear additivity model:
Revised: Finally, both effects are combined with a nonlinear additivity model:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where C is set to 0 .
Revised: where C is set to 0 .

Original: 3 and determines the overlapping effect.
Revised: 3 and determines the overlapping effect.

Original: For color images, the final RGB heatmap is H JND =
Revised: For color images, the final RGB heatmap is H JND =

Original: [ α R H, α G H, α B H ] , where ( α R , α G , α B ) are inversely proportional to the mixing coefficients for the luminance: ( α R , α G , α B ) = 0 .
Revised: [ α R H, α G H, α B H ] , where ( α R , α G , α B ) are inversely proportional to the mixing coefficients for the luminance: ( α R , α G , α B ) = 0 .

Original: A. C OMPARISON WITH ℓ ∞ C ONSTRAINT E MBEDDING
Revised: A. C OMPARISON WITH ℓ ∞ C ONSTRAINT E MBEDDING

Original: Figure 7 shows the same image activated using either the ℓ ∞ constraint (commonly used in the adversarial attack literature) or our perceptual constraint based on the JND model explained above.
Revised: Figure 7 shows the same image activated using either the ℓ ∞ constraint (commonly used in the adversarial attack literature) or our perceptual constraint based on the JND model explained above.

Original: Even with very small ε ( 4 over 255 in the example bellow), the perturbation is visible especially in the flat regions of the images, such as the sea or sky.
Revised: Even with very small ε ( 4 over 255 in the example bellow), the perturbation is visible especially in the flat regions of the images, such as the sea or sky.

Original: Laidlaw et al.
Revised: Laidlaw et al.

Original: (2021) also show that the ℓ ∞ is not a good perceptual constraint.
Revised: (2021) also show that the ℓ ∞ is not a good perceptual constraint.

Original: They use the LPIPS loss (Zhang et al., 2018) as a surrogate for the HVS to develop more imperceptible adversarial attacks.
Revised: They use the LPIPS loss (Zhang et al., 2018) as a surrogate for the HVS to develop more imperceptible adversarial attacks.

Original: Although a similar approach could be used here, we found that  the LPIPS did not capture CM and LA as well as the handcrafted perceptual models present in the compression and watermarking literature.
Revised: Although a similar approach could be used here, we found that at this small level of image distortion the LPIPS did not capture CM and LA as well as the handcrafted perceptual models present in the compression and watermarking literature.

Original: [Figure]
Revised: [Figure]

Original: B E XPERIMENTS D ETAILS
Revised: B E XPERIMENTS D ETAILS

Original: This section describes the details omitted in experimental sections.
Revised: This section describes the details omitted in experimental sections.

Original: B. D ATASET
Revised: B. D ATASET

Original: The dataset DISC 2021 was designed for the Image Similarity Challenge (Douze et al., 2021) and can be downloaded in the dataset webpage: https://ai.facebook.com/datasets/disc21-dataset/.
Revised: The dataset DISC 2021 was designed for the Image Similarity Challenge (Douze et al., 2021) and can be downloaded in the dataset webpage: https://ai.facebook.com/datasets/disc21-dataset/.

Original: We want to test performance on edited versions of activated images but in DISC query set transformations are already applied to images.
Revised: We want to test performance on edited versions of activated images but in DISC query set transformations are already applied to images.

Original: Therefore the query set cannot be used as it is.
Revised: Therefore the query set cannot be used as it is.

Original: We create a first test set “Ref10k” by selecting the 10k images from the reference set that were originally used to generate the queries (the “dev queries” from the downloadable version).
Revised: We create a first test set “Ref10k” by selecting the 10k images from the reference set that were originally used to generate the queries (the “dev queries” from the downloadable version).

Original: We also re-create a query set “Query50k”.
Revised: We also re-create a query set “Query50k”.

Original: To be as close as possible, we use the same images that were used for generating queries in DISC.
Revised: To be as close as possible, we use the same images that were used for generating queries in DISC.

Original: Edited images are generated using the AugLy library (Papakipos & Bitton, 2022), following the guidelines given in the ”Automatic Transformations” section of the
Revised: Edited images are generated using the AugLy library (Papakipos & Bitton, 2022), following the guidelines given in the “Automatic Transformations” section of the

Original: DISC paper.
Revised: DISC paper.

Original: Therefore, the main difference between the query set used in our experiments and the original one is that ours do not have manual augmentations.
Revised: Therefore, the main difference between the query set used in our experiments and the original one is that ours do not have manual augmentations.

Original: B. T RANSFORMATIONS SEEN AT TEST TIME
Revised: B. T RANSFORMATIONS SEEN AT TEST TIME

Original: They cover both spatial transformations (crops, rotation, etc.), pixel-value transformations (contrast, hue, jpeg, etc.) and “everyday life” transformations with the AugLy augmentations.
Revised: They cover both spatial transformations (crops, rotation, etc.), pixel-value transformations (contrast, hue, jpeg, etc.) and “everyday life” transformations with the AugLy augmentations.

Original: All transformations are illustrated in Fig.
Revised: All transformations are illustrated in Fig.

Original: The parameters for all transformations are the ones of the torchvision[Table] library (Marcel & Rodriguez, 2010), except for the crop and resize that represent area ratios.
Revised: The parameters for all transformations are the ones of the torchvision library (Marcel & Rodriguez, 2010), except for the crop and resize that represent area ratios.

Original: For the
Revised: For the

Original: Gaussian blur transformation we use alternatively σ , the scaling factor in the exponential, or the kernel size k b (in torchvision k b = ( σ − 0 .
Revised: Gaussian blur transformation we use alternatively σ , the scaling factor in the exponential, or the kernel size k b (in torchvision k b = ( σ − 0 .

Original: The “Random” transformation is the one used to  develop the 50k query set.
Revised: The “Random” transformation is the one used to [Table] develop the 50k query set.

Original: A series of simple 1-4 AugLy transformations are picked at random, with skewed probability for a higher number.
Revised: A series of simple 1-4 AugLy transformations are picked at random, with skewed probability for a higher number.

Original: Among the possible transformations, there are pixel-level, geometric ones, as well as embedding the image as a screenshot of a social network GUI
Revised: Among the possible transformations [Figure] there are pixel-level, geometric ones

Original: [Figure]
Revised: [Figure]

Revised: as well as embedding the image as a screenshot of a social network GUI.

Original: C M ORE E XPERIMENTAL R ESULTS C. D ETAILED METRICS ON DIFFERENT IMAGE TRANSFORMATIONS
Revised: C M ORE E XPERIMENTAL R ESULTS C. D ETAILED METRICS ON DIFFERENT IMAGE TRANSFORMATIONS

Original: On Fig. 8, we evaluate the average R @ 1 over the 10k images from the reference dataset.
Revised: On Fig. [Figure] we evaluate the average R @ 1 over the 10k images from the reference dataset.

Original: The experimental setup is the same as for Tab. 

Revised: The experimental setup is the same as for Tab.

Original: but a higher number of transformation parameters  are evaluated.
Revised: but a higher number of transformation parameters [Figure] are evaluated.

Original: As expected, the higher the strength of the transformation, the lower the retrieval performance is.
Revised: As expected, the higher the strength of the transformation, the lower the retrieval performance is.

Original: However, the decrease in performance is significantly reduced when evaluating activated images.
Revised: The decrease in performance is significantly reduced with activated images.

Original: C. A DDITIONAL ABLATIONS
Revised: C. A DDITIONAL ABLATIONS

Original: Speeding up the optimization In our experiments, the optimization is done using 10 iterations of gradient descent, which takes approximately 40ms/image.
Revised: Speeding-up the optimization. In our experiments, the optimization is done using 10 iterations of gradient descent, which takes approximately 40ms/image.

Original: If the indexation time is important (often, this is not the case and only the search time is), it can be reduced at the cost of some accuracy.
Revised: If the indexation time is important (often, this is not the case and only the search time is), it can be reduced at the cost of some accuracy.

Original: We activated 10k reference images, with the same IVF-PQ indexed presented in Sec.
Revised: We activated 10k reference images, with the same IVF-PQ indexed presented in Sec.

Original: 5.2 with only one step of gradient descent with a higher learning rate.
Revised: 5.2 with only one step of gradient descent with a higher learning rate.

Original: Activation times are computed on average.
Revised: Activation times are computed on average.

Original: The R @ 1 results in Tab. 5 indicate that the speed-up in the image optimization has a small cost in retrieval accuracy.
Revised: The R @ 1 results in Tab. 5 indicate that the speed-up in the image optimization has a small cost in retrieval accuracy.

Original: Specifically, it reduces the R @ 1 for unedited images.
Revised: Specifically, it reduces the R @ 1 for unedited images.

Original: The reason is that the learning rate is too high: it can cause the representation to be pushed too far and to leave the indexing cell.
Revised: The reason is that the learning rate is too high: it can cause the representation to be pushed too far and to leave the indexing cell.

Original: This is why a higher number number of steps and a lower learning rate are used in practice.
Revised: This is why a higher number number of steps and a lower learning rate are used in practice.

Original: If activation time is a bottleneck, it can however be useful to use less optimization steps.
Revised: If activation time is a bottleneck, it can however be useful to use less optimization steps.

Original: [Table]
Revised: [Table]

Original: Data augmentation at indexing time and EoT Expectation over Transformations (Athalye et al., 2018) was originally designed to create adversarial attacks robust to a set of image transformations.
Revised: Data augmentation at indexing time and EoT. Expectation over Transformations (Athalye et al., 2018) was originally designed to create adversarial attacks robust to a set of image transformations.

Original: We follow a similar approach to improve robustness of the marked image against a set of augmentations T .
Revised: We follow a similar approach to improve robustness of the marked image against a set of augmentations T .

Original: At each optimization step, we randomly sample A augmentations { t i } Ai =1 in T and consider the average loss: L f = (cid:80) Ai =1 L ( I, t i ; I o ) /A .
Revised: At each optimization step, we randomly sample A augmentations { t i } Ai =1 in T and consider the average loss: L f = (cid:80) Ai =1 L ( I, t i ; I o ) /A .

Original: In our experiments, T encompasses rotations, Gaussian blurs, color jitters and a differentiable approximation of the JPEG compression Shin & Song (2017).
Revised: In our experiments, T encompasses rotations, Gaussian blurs, color jitters and a differentiable approximation of the JPEG compression Shin & Song (2017).

Original: A is set to 8 and we always take the un-augmented image in the chosen set of augmentations.
Revised: A is set to 8 and we always take the un-augmented image in the chosen set of augmentations.

Original: We activated 10k reference images, with the same IVF-PQ as Sec.
Revised: We activated 10k reference images, with the same IVF-PQ as Sec.

Original: 5.2 with or without using
Revised: 5.2 with or without using

Original: EoT.
Revised: EoT.

Original: Table 6 shows the average R @ 1 performance over the images submitted to different transformations before search.
Revised: Table 6 shows the average R @ 1 performance over the images submitted to different transformations before search.

Original: EoT brings a small improvement, specifically on transformations where base performance is low ( e.g . rotation or crops here).
Revised: EoT brings a small improvement, specifically on transformations where base performance is low ( e.g . rotation or crops here).

Original: However, it comes at a higher computational cost since each gradient descent iteration needs A passes through the network, and since fewer images can be jointly activated due to GPU memory limitations (we need to store and back-propagate through A transformations for every image).
Revised: However, it comes at a higher computational cost since each gradient descent iteration needs A passes through the network, and since fewer images can be jointly activated due to GPU memory limitations (we need to store and back-propagate through A transformations for every image).

Original: If the time needed to index or activate an image is not a bottleneck, using EoT can therefore be useful.
Revised: If the time needed to index or activate an image is not a bottleneck, using EoT can therefore be useful.

Original: Otherwise, it is not worth the computational cost
Revised: Otherwise, it is not worth the computational cost.

Original: [Table]
Revised: [Table]

Revised: D A CTIVE I NDEXING VS . W ATERMARKING

Revised: Discussion.

Revised: Watermarking and active indexing both modify images for tracing and authentication, however there are significant differences between them.

Revised: Watermarking embeds arbitrary information into the image.

Revised: The information can be a message, a copyright, a user ID, etc.

Revised: In contrast, active indexing modifies it to improve the efficiency of the search engine.

Revised: Watermarking also focuses on the control over the False Positive Rate of copyright detection, i.e .

Revised: a bound on the probability that a random image has the same message as the watermarked one (up to a certain distance).

Revised: Although watermarking considers different settings than indexing methods, it could also be leveraged to facilitate the re-identification of near-duplicate images.

Revised: In this supplemental section, we consider it to address a use-case similar to the one we address in this paper with our active indexing approach.

Revised: In this scenario, the watermark encoder embeds binary identifiers into database images.

Revised: The decoded identifier is then directly mapped to the image (as the index of a list of images).

Revised: Experimental setup.

Revised: In the rest of the section, we compare active indexing against recent watermarking techniques based on deep learning.

Revised: • For indexing, we use the same setting as in Sec.

Revised: 5.1 (IVF-PQ index with 1M reference images).

Revised: When searching for an image, we look up the closest neighbor with the help of the index.

Revised: • For watermarking, we encode 20 -bit messages into images, which allows to represent 2 10 ≈ 10images (the number of reference images).

Revised: When searching for an image, we use the watermark decoder to get back an identifier and the corresponding image in the database.

Revised: Like before, we use R @ 1 as evaluation metric.

Revised: For indexing, it corresponds to the accuracy of the top-1 search result.

Revised: For watermarking, the R @ 1 also corresponds to the word accuracy of the decoding, that is the proportion of images where the message is perfectly decoded.

Revised: Indeed, with 20 -bit encoding almost all messages have an associated image in the reference set, so an error on a single bit causes a mis-identification (there is no error correction 1 ).

Revised: We use two state-of-the-art watermarking methods based on deep learning: SSL Watermarking (Fernandez et al., 2022), which also uses an adversarial-like optimization to embed messages, and HiDDeN (Zhu et al., 2018), which encodes and decodes messages thanks to Conv-BN-ReLU networks.

Revised: The only difference with the original methods is that their perturbation δ is modulated by the handcrafted perceptual attenuation model presented in App. A.

Revised: This approximately gives the same image quality, thereby allowing for a direct comparison between active indexing and watermarking.

Revised: Results.

Revised: Tab.

Revised: 7 compares the R @ 1 when different transformations are applied before search or decoding.

Revised: Our active indexing method is overall the best by a large margin.

Revised: For some transformations, watermarking methods are not as effective as passive indexing, yet for some others, like crops for HiDDeN, the watermarks are more robust.

Original: [Table]
Revised: [Table]

Original: D M ORE Q UALITATIVE R ESULTS
Revised: E M ORE Q UALITATIVE R ESULTS

Original: Figure 10 gives more examples of activated images from the DISC dataset, using the same parameters as in Sec. 5.2.
Revised: Figure 10 gives more examples of activated images from the DISC dataset, using the same parameters as in Sec. 5.2.

Original: The perturbation is very hard to notice (if not invisible), even in flat areas of the images because the perceptual model focuses on textures.
Revised: The perturbation is very hard to notice (if not invisible), even in flat areas of the images because the perceptual model focuses on textures.

Original: We also see that the perturbation forms a regular pattern.
Revised: We also see that the perturbation forms a regular pattern.

Original: This is due to the image (bilinear) resize that happens before feature extraction.
Revised: This is due to the image (bilinear) resize that happens before feature extraction.

Original: Figure 9 gives example of an image modified for several values of perturbation strength α (for instance, for α = 20 the image has PSNR 27 dB and for α = 1 the image has PSNR 49 dB).
Revised: Figure 9 gives example of an image activated at several values of perturbation strength α of Eq.(for instance, for α = 20 the image has PSNR 27 dB and for α = 1 the image has PSNR 49 dB).

Original: The higher the α , the more visible the perturbation induced by the activation is.
Revised: The higher the α , the more visible the perturbation induced by the activation is.

Original: Even with low PSNR values ( < 35 dB), it is hard to notice if an image is activated or not.
Revised: Nevertheless, even with low PSNR values ( < 35 dB), it is hard to notice if an image is activated or not.

Original: [Figure]
Revised: [Figure]