Lognormal Mutations and their Use in Detecting Surreptitious Fake Images
Authors: Olivier Teytaud
Mariia Zameshina
Tom Sander
Pierre Fernandez
Furong Ye
Laurent Najman
Thomas Bäck
Ismail Labiad
Venue: Accepted by TMLR ()

Abstract: In many cases, adversarial attacks against fake detectors employ algorithms specifically crafted for automatic image classifiers.
These algorithms perform well, thanks to an excellent ad hoc distribution of initial attacks. 
However, these attacks are easily detected due to their specific initial distribution. Consequently, we explore alternative black-box attacks inspired by generic black-box optimization tools, particularly focusing on the \lognormal{} algorithm that we successfully extend to attack fake detectors. 
Moreover, we demonstrate that this attack evades detection by neural networks trained to flag classical adversarial examples. 
Therefore, we train more general models capable of identifying a broader spectrum of attacks, including classical black-box attacks designed for images, black-box attacks driven by classical optimization, and no-box attacks.
By integrating these attack detection capabilities with fake detectors, we develop more robust and effective fake detection systems.

Published in Transactions on Machine Learning Research (02/2025)
Log-normal Mutations and their Use in Detecting Surrepti- tious Fake Images
Ismail Labiad Meta, PATR
ilabiad@meta.coom com
Thomas Bäck
t.h. w. www.beec@@iaceseedddeaiinnoooooo
LIACS, Leiden University, The Netherlands
Pierre Fernandez
pfa@meta. com
Meta, PATR
Laurent Najman
dourentt.ajmann@esiee..
Unio Gustave Biffel, CNRS, LIGM, France
Tom Sander
tomsander@mmeta.com com
Meta, FAIR
Furong Ye
f.ye @iiaacsseiieennnnio.
LIACS, Leiden University, The Netherlands
Mariia Zameshina
marid..ameehhin@@eiii.ff
Univ Gustave Fiffel, CNRS, LIGM, France
Olivier Teytaud
oteytaud@meta.comm
Meta, PAIR
Reviewed on OpenReview: https: openreview. net/ forum? id- ORJvZY0h60
Abstract
In many cases, adversarial attacks against fake detectors employ algorithms specifically crafted for automatic image classifiers. These algorithms perform well, thanks to excel- lent ad hoc distribution of initial attacks. However, these attacks are easily detected due to their specific initial distribution. Consequently, we explore alternative black-box attacks inspired by generic black- box optimization tools, particularly focusing on the log-normal algorithm that we successfully extend to attack fake detectors. Moreover, we demonstrate that this attack evades detection by neural networks trained to flag classical adversarial examples. Therefore, we train more general models capable of identifying broader spec- trum of attacks, including classical black-box attacks designed for images, black-box attacks driven by classical optimization, and no-box attacks. By integrating these attack detection capabilitics with fake detectors, we develop more robust and effective fake detection systems.
1 Introduction and outline
Due to the recent progress in AI-based image generation (oobacch et al. 2022 now casily available on internet, fake images (i.e. images created by artificial intelligence) are invading social networks. Fortunately, various fake image detectors (fake detectors), based on watermarking or supervised fake detection, have been proposed for detecting such fake images.
While new fake detectors are proposed, we also have more and more attacks targeted to make fake detectors fail. In the context of fake image detection, adversarial attacks such Square Attack (SA) (Andriushchenko

---

Published in Transactions on Machine Learning Research (02/2025)
et al., 2020) no-box techniques like diffusion-based purifiers (Saberi et al. 2024 can be used to subtly alter lake images while enabling them to effectively bypass certain fake detectors. Black-box attacks Guo et al. 2019 Biggio et al.) 22013) are imperceptible modifications of an image aimed at deceiving a classifier typically, a black-box attacked fake image becomes undetected by a fake detector based on supervised machine learning. No-box attacks are different: they have no prior knowledge on the model, and proceed by denoising: they typically remove watermarks, and therefore they deceive detectors based on watermarks. Therefore, detecting these attacks is necessary for enhancing the reliability of fake detectors. In the present paper, analyze lesser known attacks inspired by the black-box optimization community: their algorithms perform well, and are sufficiently different for being undetected by detectors trained on classical attacks o In order to increase the diversity of black-box attacks and make detectors more robust, we consider at- tacks based on generic black-box optimization methods, in particular evolutionary methods. Over the past decades, itcrative optimization heuristics like Evolutionary Algorithms (EAs) have proven effective in tack- ling challenging optimization problems. While specialized methods have emerged for specific domains- such as Genetic Algorithms for discrete optimization and Evolution Strategies (ES) for continuous optimiza- tion-various adaptations of these techniques have been proposed and successfully applicd across diverse domains. Regarding the topic of borrowing ideas across different domains, there exists much work in ex- changing ideas between discrete and continuous optimization domains. For example, variants of Covariance Matrix Adaptation ES (CMA-ES) and Differential Evolution (DE) have been proposed for specific prob- lems (Hansen & Ostermeier 2003; Hamano et al., 2022: Das et al. 2016). More recently, a study has investi- gated the ways of discretizing CMA-ES and its performance on different discrete BBOB problems (Thomaser ct al. 2023)- In this work, we investigate the advantage of borrowing an idea from self-adaptive pscudo- Boolean optimization algorithm to the continuous domain. One of EAs' key algorithmic components is the utilization of a probability distribution for generating new search points, including an iterative adaptation of this distribution based on the objective function values of the newly generated solutions (Doerr et al.) 2020: Doerr& & Neumann 2020). A wide range of such update strategies has been proposed over the past decades, for example, for multivariate Gaussian distributions in the context. of covariance matrix adaptation in evo- lution strategies for continuous optimization problems (Hansen & Ostermeier 2003) and for binomial and a varicty of other mutation strength distributions in the context of mutation rates in evolutionary algorithms for pseudo-Boolee
optimization (Doerr et al. 2020 Among those adaplation methods for the mulation strength in pseudo-Boolean Boolean optimization, the so- salled log-normal mutation was developed almost thirty years ago as a method for allowing the mutation strength to quickly shift from exploration to exploitation [Back & Schütz 1995 Bäck & Schützz 1996 Kruisselbrink et al. 2011). As shown in (Doerr et al. 2020 on a set of 23 pseudo-Boolean test. functions, log-normal mutation shows an empirical cumulative distribution functions (ECDF) performance across all 23 functions that are very close to the pers-perforning algorithm. In this paper, we compare the log- normal (LN) mutation with various algorithms for continuous optimization problems selected from the Nevergrad benchmarks (Rapin & Teytaud. 2018). Experimental results indicate that the log-normal strategy is competitive for problems that are particularly difficult, i.c., multi-modal and highly-deceplive. Inspired by the mentioned benchmarking results, we test the log-normal mutation for a practical scenario, namely the attack of fake detectors. These methods are not necessarily better than e.g. SA (Andriushchenko et al., 2020): but they are completely different, not specialized on images, and free from typical artifacts from specialized methods (Fig. [.1]p making them undetected by existing detectors. We then propose new detectors, for mitigating the risk associated to those methods based on generic black-box optimization.
Outline: Section 2 presents the state of the art in fake detectors (section 2.1 and Black-Box Optimization (BBO, section 2.2 Section 3 present our tools: section 3.1 for the log-normal mutations and section 3.2 for the extension of log-normal normal to continuous domains. Then section presents preliminary experimental results, including an ablation (section 4.1), results on the Nevergrad benchmark suite (Section 4.2 and discussing specific results (Section 4.3 these preliminaries justify the choice of LogNormal as a tool for our black-box attacks of fake detectors, satisfying both (i) reasonably effective (ii) completely different from usual attacks. Section 5 focuses on the application to fake detectors: comparing various algorithms on this application, we
All data collection and experiments were conducted on university servers by people with corresponding a affiliations.
2

---

Published in Transactions on Machine Learning Research (02/2025)
observe particularly good performance of log-normal mutation. Section combines new detectors and existing ones for improved performance.
Contributions: (I) We present an extension of log-normal mutation (usually applied in the context of discrete domains) to arbitrary discrete, continuous and mixed search spaces (Section 3.2). (2) We present
a comparison between log-normal mutation and a state-of-the-art algorithm selector (Meunier et al. 2022
illustrating a particular strength of log-normal for deceptive and multimodal problems (Section 4.2 4.3 (3)
We show that log-normal mutation is particularly well suited for attacking fake image detectors:
normal mutation performs well (Section 5.1 and the created fake images go undetected by detectors based on existing attacks (Section 5.2.2. For converting the attacks into defense, we then add a detector of these attacks (Section 5.2.2 and study its robustness (Section 6/.
2 State of the art
In this section, we briefly discuss the state-of-the-art in fake detection methods, to set the stage for using those as a specific application domain for black-box-pllimization (section [.1]. Then, we introduce the general setup of the black-box optimization problem, and the set of algorithms that are used in the empirical comparison presented in this paper (section 2.2
2.1 Fake detectors
Due to the many Al-generated images on internet, it becomes important to be able to detect them. DIRE (Wang et al. 2023 is method to detect images generated by Latent Diffusion Models (LDM) (Rom-) bach al. 2022). focuses on the error between an input image and its reconstruction counterpart using a pre-trained diffusion model. This method is based on the observation that the generated images can be ap- proximately reconstructed by diffusion model, while real images cannot. Several papers already mentioned artifacts making DIRE unreliable in a real-world sccnario, in particular the impact of the image format on the prediction ion (Moskowitz et al., 2024). Another line of work (Coccomini et al. 2023) explores the lask using simpler, more traditional machine learning algorithms like Multi-Layer Perceptrons (MLPs), features extracted by Contrastive Language-Imare Pretraining (CLIP, (Radford et al. 2021)), or traditional Convo- lutional Neural Networks (CNNs). Generalization to unseen image generators known as critical issue for such supervised learning approaches: Universal Fake Detector (Ojha et al., 2023) uses linear model trained on top of a generic feature extractor, for the sake of generalization and transfer, and GenDet (Zhu et al. 2023) uses an adversarial method aimed al solving the problem of unknown fake generators. Another method that helps to detect LDM -egegeerrt images is the Local Intrinsic Dimensionality (LID) method [Lorenz et al., )023). LID is employed to estimate the intrinsic dimensionality of a learned representation space, measuring the average distance between a point and its neighboring points. This approach is vital in characterizing the distinct properties of adversarial and natural samples in the latent space of a classifier. The multiLID method (Lorenz al.) 2023)- extends this concept, combining locally discriminative informa- tion about growth rates in close proximity, which proves effective in detecting adversarial examples as well as images generated by diffussin models. Watermarking (Cox et al. 2007) and on-device certification (Nagm) ett al. 2021 are alternative methods for distinguishing between fake and real images. Watermarking is de- signed to create imperceptible alterations to the images. These alterations, however, can be detected and decoded with the appropriate algorithms. The primary purpose of watermarking is to ensure traceability of the images back the model. Many of the current generation models already use built-in watermarking techniques. For instance, in Stable Diffusion (Rombach et al., 2022) the authors use Discrete Wavelet Trans- form (DWT) together with Discrete Cosine Transform (DCT) (Ai-Haj 2007]. While these transformations provide some protection from minor generative image alterations, they not resistant, for example, to image resizing. Furthermore, one can casily disable the watermarking in the Stable Diffusion code. More advanced watermarking methods, as proposed, for example, by Fernandez el al.] (2023), merge watermarking into the generation process. Watermarking methods are brittle to adversarial attacks. They include no- no-box attacks which do not need any knowledge about the method, c.g., using perceptual anto-cneoders (Fernan- dez et al., (202)) or noising/demt /denoiindnn with diffusion model (Nie al. (002)): and white-box or black-box attacks (in particular, SA) which employs the watermark extractor or APT access to it (Jiang et. al.
3

---

Published in Transactions on Machine Learning Research (02/2025)
2023
Fortunately, there are effective detectors for both no-box attacks and for SA. We investigate how generic black-box optimization methods can be turned into adversarial attacks that are not detected by these detectors: therefore, we need detectors for this family of attacks.
2.2
Black-box optimization
An unconstrained optimization problem can be generally formulated follows, min f(x), E Q. where Ω denotes the search space, f Ω - I and m is the number of objectives. Note that we consider m = 1 in this work, and without loss of generality, we assume minimization problem. According to the domain of Ω we can consider problems as continuous optimization when Q C IR", as discrete optimization when Q C Z", and Boolean optimization (PBO) when = {0, 13", where n is the dimensionality of the problem. We deal with black- box optimization (BBO), in which algorithms can not obtain the exact definition of the objective function f and constraint definitions regarding the structure of f. Evolutionary computation has been widely applied to solve BBO. For example, Evolution Strategies (ES). Differential Evolution (DE), etc., have achieved success in solving continuous BBO problems (Bäck et al. 2023 The variants of these methods have
pseudo-Boolean
also been applied for discrete BBO based on relaxation to a continuous problem (Pan et al. 2008 Hamano
et al. )022). Moreover, EAs have been well-studied for pseudo- Boolean optimization (PBO) (Doerr et al. 2020 Some strategies have been commonly applied with specific adjustments when solving different types of optimization problems. A recent study has investigated the performance of a discretized Covariance Matrix Adaptation Evolution Strategy (CMA-ES), addressing the adaptation of continuous optimization algorithms
for the discrete domain [Tomaseer et al. 2023
In this work, we work on continuous optimization by utilizing discrete optimization algorithms. Specifically, we investigate utilizing the techniques of the (1+)) EAs (where À refers to the parallelism) that are designed for PBO for continuous optimization (Doerr et al. 2020 The (1 + A) EAs flip a number e of variables to generate À new solutions from a single parent solution iteratively, and self-adaptive methods have been proposed to adjust the value of online, essentially detecting the optimal number of variables to be altered dynamically. We consider various black-box optimization methods, which can all be found in (Rapin & Teytand| 2018). We include many algorithms, specified in Section D.1 We selected, for readability, sample of methods covering important baselines (such as random search), one representative method per group of related methods, and the overall best methods. We refer to (Rapin & Teytaud 2018) for the details about other algorithms, and to Beyer (2001) and references in Section D.1 for more details about generic optimization algorithms.
3 Introducing the LogNormal method as a different black-box method
The present section introduces the LogNormal algorithm and its adaptation to continuous settings. The next section will then validate it experimentally, so that Sections and G will use the LogNormal method (and corresponding defenses) in our context of fake detection.
3.1 Log-normal mutations
We introduce in this section the general framework of the (1+A) EA with log-normal mutation, which was proposed for the pseudo-Boolens optimization task, and illustrate a straightforward generalization of this algorithm to continuous and integer domains.
The log-normal mutation was first described in [Back & Schütz 1995; Bäck Schütz 1996), , later refined in KKrisselbriik et al.) 2011). It stands out for its robust performance across various problem landscapes (Doerr et al. 2020). Although it may not surpass other self-adjusting methods, such the two-rate (Doerr et al, 2019b and the normalized bit mutation (Ye et al. 2019), which are good at converging to small mutation strength f=1 1 for the classic theory-oriented problems OneMax, LeadingOnes, etc., log-normal mutation consistently delivers competitive results for complex practical problems such as Ising models and Maximum Independent Vertex Set (Doerr al. 2020). Following the sctup recent work (Doerr al.) (2220) on pscudo- Boolean optimization, we provide concise formulation of a (1 + X) EA with the log- normnolal mutation in Algorithm 1 The algorithm starts from a randomly initialized point (line and generates À offspring by the
4

---

Published in Transactions on Machine Learning Research (02/2025)
Algorithm 1: (1 2) EAtog-n with log-normal mutation.
Input: given problem f: Ω + I,, where n denotes the dimensionality of the problem, an initial value of the mutation rate p (0, 1), population size À > 0, and learning rate 0.22;
Initialization: Sample E Ω u.a.r. and evaluate f(æ);
Optimization: for f = 1, 2, 3, do
for i == I,...,A do ~ 10,
5
p p (1
1- exp(7 9)
Sample (() ~ (n, (n))))
7
create y() MUTATE (1)), xx); evaluate f(y())
// See Alg
min {i flyli) ff (1)) [AB]] + p x² arg
10
[(
// ties broken by favoring the smallest index
12
if f(x')) S f(x) then x + xx;;
13
Output: v, (x)
Algorithm 2: MUTATE((,x). Note that the uniform random distribution is applied to the domain of the corresponding variable: in the case of our attacks (which always bounded for L*),, this means randomly drawing in 1-0.03, 0.03]. Similarly, the general purpose benchmarks in appendix, the mutation uses the domain of the corresponding variable in the corresponding problem.
Input: a solution x E X Xr, and the mutation strength E E (1,2....,,)) n), where n is the dimensionality of the problem;
Sample l pairwise different positions f1....,44 [n] u.a.r.;
y <-;; -;;
for 1, do repeat
I 1 ~ u(x;)) until k # Digj
// Sample k u.a.r. from domain X;
Vi, + k;
Output: y;:
MUTATE function in the for-loop (lines We denote the mutation strength é as the number variables to be altered, and this number is generated for each newly generated solution candidate, i.e., offspring, by first mutating the mutation rate p. In practice, p mutated according to log-normal distribution (line 6 where A/ 1) is normally distributed random variable with expectation zero and standard deviation one, from which is sampled in line 5). and this rule maintains that the median of the distribution of new mutation rates p()) is equal to the current mutation rate p. p(9) then determines the value of ( (line 7. and the corresponding offspring is created and evaluated in line S Specifically, g(6) is sampled from a binomial distribution (n, (+) (line 7), where n is the dimensionality,, and the sampling repeated until obtaining ( > 0. In case of ties the objective function values, the mutation rate used for the first best of the newly generated solutions (line g is taken for the next iteration (line I0]. The best solution x is updated in line 12 if the best. offspring equal to better than the current parent solution.
3.2 Modifications of the original normnl algorithm: adaption to the continuous setting
Note that in Algorithm [ we do not specify the domain Ω æ. The log-normal mutation has been commonly applied for the pscudo-Booles optimization, i.c., Ω = (0, 1]", and the MUTATE-operator flips 7(0) bits that are selected uniformly random. In this work, we introduce a generalization of the MUTATE-operator, presented in Algorithm g to extend the log-normal mutation to ht domains. This generalized MUTATE: is
5

---

Published in Transactions on Machine Learning Research (02/2025)
Name
Initial
Name
Tnitial p
Lognormal (Standard sctting)
0.2
12
SmallLognornal
0.2
BigLognormal
0.2
120
XLognormal
0.8
12
Hogernggn
0.2
1200
X
0.8
4
OLN(Combined with bandits)
0.2
12
Table 1: Variants of tog-normal mutations used in the experiments. OLN is combination with bandits, for managing reevaluations in noisy optimization context.
applicable for arbitrary search domains Q = X X, where X E ((0,1),Z,R,N) It samples a new value that is distinct from the current one random from the respective domain for each of & variables, which are selected (uniformly at random) for mutation. This technique has also been adopted by Nevergrad (Rapin) & Teytaud 2018), which automatically adapts algorithms designed for continuous domains to also work for discrete domains and vice versa.
3.3 The LogNormal attack applied to fake detectors: difference with Square Attacks
In the present Section, we clarify what the LogNormal attack is doing on an image. The adversarial attack consists in finding such that ++ is classified as non-fake, whereas I is classified fake. We maintain -0.03 < Ei.j. < 0.03 (corresponding to pixel (i,j) for channel k),, i.e., our attack is bounded in L the same way SA and other attacks. Note that the naming LogNormal corresponds to the tool used by the algorithm for choosing the mutation rate (as detailed in Section 3.1): this does not imply that the noise added to the image follows a log-normal probability distribution. As explained in Scction 3.2 the perturbation affects a subset of pixels, with an added value 6i.j randomly uniformly drawn in [-0.03, 0.03]. Compared to SA, note that the algorithm does not generate the typical horizontal or vertical artifacts corresponding to the squares (Fig. 5.1 : this explains that it is more difficult to identify, both by supervised learning and by human investigation.
4 Preliminaries: experimental results on the Nevergrad Black-box optimization benchmarks
The present section shows that the LogNormal algorithm performs reasonably well, including in the unex- pected continuous settings, in particular for the black-box optimization benchmarks from Nevergrad that are somewhat (as far as possible for generic benchmarks unrelated to vision) related to our image context.
4.1 Parameter settings & Ablation
While the log-normal mutation control mutation rates of the (1+A) EA online, Algorithm stiil comprises three hyperparameterss, i.e., the initial value of population size X, and 7, that can affect the a algorithm's performance. We set 7 = 0.22 following the suggestion in previous studies (Kruisselbrink et al. 2011 Doerr et al. 2020). For the other two hyperparameters, we test two values 0.2 and 0.8 for and various population sizes The detailed combinations are presented in Table
We denote log-normal as the standard setting. BigLognormal experiments are performed with an incrcase population size (respectively, large increase for Lugnoogn,,,,,, and decrease for Smalllognormal) XLog- normal obtains a greater initial mutation rate. OLN (Optimistic Lognormal) experiments the combination with Optimism as performed in Nevergrad for making deterministic algorithms compatible with noisy opti- mization (see appendix D.3
We compare the settings of Algorithm [ with several algorithms, such as random search, CMA, and the anisotropic adaptive algorithm (Doerr al.) 2016 2017) provided by Nevergrad. log-normal with the stan-
dard parameterization from (Bäck & Schütz 1996) performs essentially well across the 14 tested problems (see Appendix F), though the "Big" variant an interesting outsider. We note that the default parametriza-

---

Published in Transactions on Machine Learning Research (02/2025)
tion of log-normal fails mainly on topology optimization (for which anisotropic methods perform great) and noisy problems (which are irrelevant for our context of attacking deterministie fake detectors, and we note that a good solution in that setting is to use a combination with optimism in front of uncertainty as proposed in (Rapin & Teytand 2018)). ). We therefore keep the standard log- normal and some "Big" counterparts for our application to Take detectors.
4.2 Robust performance of nog-normal
The log-normal algorithm has alrcady shown competitive results in existing benchmark studies (Doerr et al., 2019b), and in the present paper we examine its performance by comparing it on extensive benchmarks provided by Nevergrad. Table [1 shows the diversity of the benchmarks in Nevergrad, where each benchmark is accompanied with list of baselines.
Since NGOpt is a "wizard" provided by Nevergrad that is tuned for selecting automatically a proper algorithm for each problem, we use it as a baseline for comparison. We run log-normal and NGOpt and proposed default methods. Tabless 2 and 12 list the rank of the log-normal algorithm and of NGOpt: Table 2 lists the problems in which the benchmark outperforms NGOpt, and Table 12 presents the results of the other benchmarks.
We observe that the log-normal algorithm can outperform NGOpt 17 out of 41 benchmarks and perform better than 50% of all algorithms on 29 benchmarks. Recall that Nevergrad contains diverse set of bench- marks, and it is well-known that we can not expect one algorithm to perform the best for all problems. The lognormal algorithm shows robust and competitive performance for the single-objective, difficult, low ratio budget/ bilgge//dimonsin /dimonsion benchmarks in Nevergrad. Of course, NGOpt is difficult to beat: it is an optimization LizarqLiu et al.] (2020), automatically selecting a black-box optimization method specifically for the dimen- sion, budget and type of variables: LogNormal can not compete in the general case as it is very naive for multi-objective, constrained stochastic contexts: this is not our purpose, we just observe that LogNormal performs well problems somewhat related to adversarial attacks.
Since the log-normal algorithm was originally proposed for discrete optimization, we first present the detailed results of log-normal variants for the two discrete benchmarks in Fig. 4 In Fig. 4 the y-axis represents the loss of algorithms for tested budgets (presented on the x-axis), and algorithm labels are annotated by their average loss for all the maximum budget between parentheses and (for checking stability) the average loss for the tested budgets excluding the largest one between brackets (if there are least three budget values). For readability, we present only the 35 best results (and the worst, for scale) for each benchmark. We can observe that the log-normal variants obtain the best performance for the two discrete benchmarks.
4.3 Competitive results for deceptive problems
Recall that in the ranks in Table 2 the log-normal algorithm performs best for the "deceptive" and "fishing" benchmarks and outperforms NGOpt on many difficult multimodal benchmarks (marked with **). We
plot in Fig. 5 the detailed results of the best of 61 algorithms on the deceptive benchmark. The deceptive benchmark combines many random translations of hard problems in many dimensionalities, including (i) a problem with a long path to the optimum, which becomes thinner and thinner close the optimum, (ii) a problem with infinitely many local minima, (iii) a problem with condition number growing to infinity as we get closer to the optimum. We note that this hard continuous benchmark, the log-normal algorithm performs well across all the lested budgets. Therefore, due to ils robust performance across number of benchmarks and particularly competitive performance for the hardest benchmarks, we utilize the log-normal mutation for the following fake detector scenarios.
5 New attacks of fake detectors
We attack fake detectors. More precisely, we add imperceptible noise e to the image x, with this noise chosen so that fake detectors fail: typically x is a fake image detected as fake by a detector D and we find a small e such that + classified as non-fake by D. Black-box algorithms can be straightforwardly applied to tatackgng a fake detector D (which returns the estimated probability D(x) that image x is fake) by
7

---

Published in Transactions on Machine Learning Research (02/2025)
Cases in which log-normal outperforms NGOpt
Problem
Rank of log-normal
Num algorithms
Rank of NGOpt
deceptive**
0
61
6
fishing**
56
miliobj-examml---any-ll
1
67
48
yatuningbbob
54
multiooj-example-//
5
69
55
multiobj-example
9
29
yeonepensmallbbob
9
79
15
nano-seq-mlluning
10
23
11
yasmallbbob
10
96
19
nano-naive-ee-mmlaunig
11
23
14
xp-pbbob
13
41
26
nano- veryseq- miteninggnninnnggg
14
26
16
pbo-reduced-suite verysmall-pholoniee verysmaal-phoooneee pbbob
16
156
20
26
64
30
33
37
35
63
42
uitrasmall-photooics22
44
83
72
Table 2: Results on benchmarks from the Nevergard benchmarking suite for which log-normal outperforms NGOpt. The rank, computed as described in Section E between and num algos - 1. We compare log-normal NGOpt. Cases which log-normal outperformed by NGOpt are in Table 12 We note that results on continuous problems are not bad, in particular for hard problems: Deceptive (which is designed for being hard), some multi-objective benchmarks (in particular the many-objective case), PBBOB which usees difficult distributions of random translations for optima, very low budget problems such YaTumingBBBB, and difficult low budget Photonics or Fishing problems. * denotes discrete problems, ** denotes single- objective problems which are highly multimodal and difficult.
defining (oss(e) D(e x)) and minimizing loss on domain D (typically 00003, 0.03]', where t is the shape of the image tensof using black-box algorithm. Our attacks (Section 5.1 consider the fake detector black-box, so we do not have gradients. Then, in Section 5.2 we create defense mechanisms by detecting various black-box altacks, showing that the log-normal attack is not detected by a detector created for a classical attack (such as Square Attack) and, therefore, needs new ad hoc detectors.
5.1 Black-box attacking fake detectors
The fake detector we consider is the universal fake detector (Ojha et al. 2023 Since the advent of ad- versarial attacks (Guo et al.] 2019 Biggio et al. 2013 it is possible to modily data (e.g. an image) so that it is misclassifidd, while remaining, for visual inspection, close to the original image for the I norm. This critical for the success of fake detectors, as fake detectors can be attacked by such imperceptible adversarial attacks. Typical adversarial attacks for images include (Goodfellow et al. 2014 Biggio et al.,
2013 Andriushchenko et. al. 2020: Jiang et. al. 2023 Chen al.] 2020 Jiang al. 2023 Our dataset of fake images is sampled from the test set provided by the authors of Ojha et al. (2023 Precisely, consider the datasets as described in Table 3 IN500- refers to the 500 first classes of ImageNet, IN500+ to the other
classes of ImageNet, G refers to Glide, PG refers to ProGAN, BG refers to BigGAN, LDM-200 is defined
in OOjha et al. 2023 In the present paper, Clean means unattacked, neither by no-box nor by black-box attacks.
Critical cases. We consider the detection of attacks (no-box or black-bog attacks) aimed at evading fake detectors. So, we particularly interested in correctly detecting attacks in two cases (1) fake images attacked for making them appear genuine and (2) real, unaltered images (clean). The other cases are less
In this case, assume that pixels in the range [0, 1]. Due to maximum values image representations we might clamp the values of
8

---

Published in Transactions on Machine Learning Research (02/2025)
Name
Description
Datasetl
2k images from PG, 2k images from BG and 1k images from LDM-200 for a total of 5k fake images. Used to benchmark variations of log-normal attacks and square attack.
Datasct2-DP
Same as Datasct1, plus 5k real images from IN500- and their purified versions by DufePure with parameter 0.1
Datasct2-IR
Same as Dataset1, plus 5k real images from IN500- and their purified versions by ImageRephrase with parameter 0.1
Dataset2-SA
Same as Dataset 1, plus 5k real images from IN500- and their attacks by SA with budget=10k and 1 = 0.01. Total = 20k images.
Dataset2-LN
Same as Dataset1, plus 5k real images from IN500- and their attacks by log-normal (algol) with budget=10k and 1 0.01. Total 20k images.
Dataset3
1K real images from IN500+ 500 fake images from PG of different classes than Dataset2 + 500 fake images from P and their attacked counterpart. Total images.
4k
Dataset3- 1, IRO.2, IR0.3
Purified versions of Dataset3 based on ImageRephrase, with parameter 0.1, 0.2, 0.3.
Dataset3-DPO.1, DP0.2, DP0.3
Purified versions of Dataset3 based on DiffPure, with parameter 0.1, 0.2, 0.3.
Dataset3-SA (L,B)
Real images from Dataset3 + fake images from Dataset3 attacked by SA with various amplitudes (L {0.01, 0.03, 0.05)) and budget (B {1000, 10000}}.
Dataset3-LN (L,B)
Real images from Dataset3 + fake images from Dataset3 attacked by LN (algol to algo5) with various L {0.01, 0.03, 0.05) and B (1000, 10000).
Dtaset4
lk fake images from Dataset3, attacked with log-normal attacks (algol).
Table 3: Datasets used in the study, Dataset3 will be used test set for the detectors trained on Dataset2 and Dataset4, will be used for testing the transfer of detectors trained on SA attacks to log-normal attacks.
critical: (1) the case of real images altacked so that they will look fake: this will usually not prevent the original images from being classified as real, making the situation less annoying. images (erroncously viewing them as attacked will not prevent fake detector from working). Therefore, besides our accuracies datasets, will present accuracies the critical part of these datasets.
(2) the case of clean fake
We first attack the fake detector with the classical SA, and then with generic black-box optimization methods from section 2.2 All our black-box attacks are bounded in 1 and in all experiments, we use the same = bound on attacks for all methods. We give a brief overview here and full details are in appendix D.2 Nevergrad provides modifiers dedicated to domains shaped as images. These modifiers can be applied to any black-box optimization algorithm. For example, the prefix Smooth means that the algorithm periodically tries to smooth its attack x, if the loss of Smooth(z) is better than the loss of x, then is replaced by Smooth(z) in the optimization algorithm. We add two additional modifiers specific from adversarial attacks. Both are inspired from SA, though our modifications have, by definition, less horizontal and vertical artifacts (Fig. 5.1. First, G (Great) means that we replace loss(æ) by loss(0.03 x sign(x)) when the allowed norm of the attack is 0.03 for the 1o norm. Second, SM (smooth) means that we replace the loss function f by f (convolve(z)). where convolve applics normal blurring with standard deviation 3/8. GSM mcans that we apply both and SM. For example, we get G, SM and SuperSmooth modifiers top of the standard log-normal method, and various methods
by applying
9

---

Published in Transactions on Machine Learning Research (02/2025)
Original
algo 1
algo 5
SA
Figure Example of attacked images. All attacks are done with budget of 10k queries and 1 0.03.
We observe vertical and horizontal artifacts in the of SA.
Algorithms
Alias algol
Algorithms
Alias
algo1
algo2
algo5
algo3
algo6
Algorithm
SA
algol
algo2
algo3
algo4
algo5
algo6
Success rate on dataset1 100%
91.0%
94.1%
64.6%
82.0%
99.2%
99.1%
Table 4: Top: Variations of the log-normal algorithm considered. Bottom: Success rate when attacking the fake detector. The success rate is computed solely on atlacks of correctly classified clean images. All atlckk have 10k budget and - 0.03. Inspired by SA, with maximum values of modifications (*G") covering area ("SM"), Algo5 performs best among LogNormal variants. Compared to SA, it is less specific from images and it has a self-adaptive mutation rate starting at high value ("Big"): the key point for us is that it performs reasonably well while the transfer from SA detectors fails.
in Tab 4 (top). Results are presented in Tab. 4 (bottom). Examples of attacked images are presented in Figure Basically, when we have a black-box access to a fake detector, we can attack it either by SA or by log-normal with budget 10k queries and 1 = 0.03.
5.2 From attack to defense: detecting attacks
A recent (rend is the denoising of watermarked images for evading the detection: whereas the watermark makes possible to detect that an image is fake, the denoised version goes undetected. However, a defense is detect such a denoising (c.g. by DiffPurc or ImageRephrasc), or other forms of attacks (such as SA). That way, our work both provides (i) new attacks and (ii) detectors for those attacks, be used as an additional step (detection of evasion) in fake detectors.
10

---

Published in Transactions on Machine Learning Research (02/2025)
Attacks
Clean
DP 0.1
DP 0.2
DP 0.3
IR 0.1
IR. 0.2
IR 0.3
FNR
5.2%
23.42%
34.4%
40.1%
11.3%
16.8%
26.2%
PSNR
NA
27.9
24.4
22.1
26.6
24.4
22.7
Table 5: The second row shows the false negative rate (FNR) at threshold of 0.5 for the universal fake detector on clean Datasetl consisting of only fake images and different purified versions of this dataset. DP stands for DiffPure while IR stands for ImageRephrase. The third row corresponds to the average PSNR.
5.2.1 No-box attacks (purifiers) and their detection
While watermarking techniques can be made robust against some image alterations such as resizing or JPEG compression as shown by Fernandez et al. (2023), they remain vulnerable to no-box purification at- tacks (Saberi et al. 2023) that destroy the hidden message. Additionally, the distribution change introduced by no-box purification can impact negatively the performance of fake detector, as shown in Table 5 We will next show that one can easily detect these attacks if we have access to the purification method. We consider two purifiers used by Saberi et al. (2024)): DiffPure based guided diffusion and ImageRephraso based on latent diffusion. We consider the strength/stees parameter in the set {0.1, 0.2, 0.3). These two purifiers lead to an average PSNR (Peak Signal-to-Noise Ratio) between 20 and 30, which is a noticeable yet acceptable quality loss. As expected, the PSNR of the distortion increases with the strongth paramcter. For detecting the no-box attacks, we train classifier, specified in table [4] on Dataset2-DP or Dataset2-IR, We experiement with both ResNet50 and SRnet (Boroumand overall. We use Dataset3 as a test set (so that the distributions of images differ) with images attacked by same purifier but with different parameters in {0.1, 0.2, 0.3). The results of the detection of latent purifiers and guided diffusion purifiers are shown in Table 6 and Table 7 We achieve less than 5% FPR and FNR across the hold-out training dataset and the critical part of testset, although the FPR is relatively higher for the full testset that can be explained by having a sort of transfer from detecting DP or IR to detecting images generated by G. In short, training on a purifier with a specific parameter allows us to detect that same purifier with different parameters and for different image distributions.
al.j 2018) and the latter performs better
5.2.2 The detection of black-box attacks
For delecting square allacks, we train a deep net using a similar setup, as the SRnet model still performs better than ResNet in this scenario. We also employ data augmentation techniques: horizontal flip, random crop and color jitter which empirically makes the model more robust to different attack parameters, while allowing us to train using only one attack parameter budget=10k and n 0.01. The training is done 2-SA. For testing, we use Dataset3-SA using different parameters than those used during training. We then test the transfer of the SA detector on Dataset4, corresponding to Log-normal attacks. Table g summarizes the results of the SA detector, for detecting SA and as a detector of normall attacks. We observe that the transfer detecting log-normal attacks very poor. So, we need to include such attacks in our training for improving the defense.
A new detector for log-normal attacks. We have seen that detectors of SA do not detect our log- normal attacks. For detecting log-normal attacks, we use the same setup in section 5.2.2 for creating new detector. Datasct2-LN is used during training with the attacked images now obtained with GSM- budget=10k and 1 = 0.01 and again, the dataset is split to 80% train, 10% test, 10% validation. For testing, we use Dataset3 with the images attacked using different variations of log-normal and paramcters (Budget and 1o0). Table g presents the results of the logmnlrmal detector: we observe that the learning was made on images attacked by algol only and we get positive results for all lor-normal variants.
11

---

Published in Transactions on Machine Learning Research (02/2025)
Dataset
FPR.
FNR.
AUC1
(same dist)
(atass22(p hold-out
0.9%
0.3%
0.99
Dataset 3, full (critical and non critical images)
Dataset3 (param=1R0.1)
23.1%
0.1%
0.96
(different dist)
Dataset3 (param=1R0.2)
23.1%
0.2%
0.96
Datasct3 param=IRO.3)
23.1%
0.2%
0.95
Dataset 3, critical part
Dataset3
(param=1Ro.1)
1.7%
0.0%
0.99
(different dist)
Datasct3
pparam-RR0.2]
1.7%
0.0%
0.99
Dataset3
param=IR003)
1.7%
0.0%
0.99
Table 6: False positive and false negative rates for purified images detection at a threshold of (0.5) alongside with the AUC score. The purified training images correspond to the latent-purifier (class "ImageRephrase") with parameter 0.1. We observe good results, in particular for critical cases.
Datasct
FPR
FNR,
AUC1
(same dist)
Dataaoo2 hold-out
1.9%
3.9%
0.99
Dataset 3, full (critical and non critical)
Dataset3 (param=DPO.1)
23.0%
3.7%
0.88
(different dist)
Dataset3 (param=DPO.2)
23.0%
4.0%
0.87
Dataset3 (param=DPO.3)
23.0%
4.2%
0.87
Dataset 3, critical part
Datasct3 (param=DPO.1)
2.2%
3.0%
0.99
(different dist)
Dataset3 (param=DP0.2)
2.2%
4.6%
0.99
Dataset3 (param=DPO.3)
2.2%
5.4%
0.99
Table 7: False positive and false negative rates for purified images detection at threshold of (0.5) alongside with the AUC score. The purified training images correspond to the guided-diffusion purifier (class "Diff- Pure") with parameter 0.1. We observe good results, in particular when we restrict the analysis to critical cases (see the specification of crit cases in section 5.1).
6 Combining detectors
We need to detect both no-box and black-box atlacks. As experiments show that other black-box attacks could be used instead of classical ones, we also need transfer among different black-box attacks. Table I0 presents a combination of detectors: we classify an image as fake if it has an estimated probability of fake/attacker greater than for least one of our base detectors. Our base detectors are (i) detector of SA (ii) the detector of CMA attacks (iii) the detector IR (ImageRephrase (Saberi et al., (024b)) (iv) the detector MBT (Minnen et al. 2018) (v) the watermark detector (vi) the UFD. On purpose, we do not include the detector of LogNormal attacks for checking that our combined detectors transfers to detecting LogNormal attacks without having seen them.
7 Conclusions
We tested log- normal mutations various benchmarks and extended it to continuous benchmarks, includ- ing fake detection tasks. Lognormal mutations perform well in some cases in the Nevergrad benchmarks, especially on: (1) PBO, a classical discrete benchmark, and some other discrete benchmarks (2) In some continuous contexts, in particular in the most difficult scenarios such as many-objective, highly multimodal, low budget benchmarks, including c.g. real world benchmarks photonics. We note that other discrcte algorithms adapted the continuous case do perform well. We note that discrete optimization methods are relevant in in optimization, when the prior (i.e. the range of reasonable values for each variable). is
12

---

Published in Transactions on Machine Learning Research (02/2025)
Dataset
FPR/
FNR
AUC1
(same dist)
Datasct2-SA hold-out
1.8%
0.5%
0.99
Datasct3-SA (B=10k, L-0.01)
0.8%
0.9%
0.99
Dataset3-Si 3-SA 1B=10k, L=0.03)
0.8%
0.0%
0.99
Dataset3-SA (B=10k, L=0.05)
0.8%
0.0%
0.99
(different dist)
D-taset3-SA (B= 1k, L-0.01)
0.8%
0.0%
0.99
Dataset3-SA (B= 1k, L=0.03)
0.8%
0.0%
0.99
Dataset3-SA (B=1k, 0.00.5)
0.8%
0.0%
0.99
(transfer to log-normal)
Datasct4
NA
79.5%
NA
Table 8: False positive and false negative rates for square attack detection at a threshold of (0.5) alongside with the AUC score. B stands for the budget used for the attack and stands for the accepted 1es distance. Last row: (clearly failed) transfer to log- normall detection.
Dataset
FPR
FNR
AUC1
(same dist)
Dataset2-LN LN hold-out
0.7%
2.1%
0.99
Datasct3-LN (algo1, B=10k, I.=0.01)
4.1%
5.4%
0.98
DaLaset3-LN (algol, B=10k, L-0.03)
4.1%
2.3%
0.99
Dataset3-LN (algol, B=10k, L=0.05)
4.1%
2.9%
0.99
(different dist)
Dataset3-LN (algo2, B=10k, L-0.03)
4.1%
4.1%
0.98
Dataset3-LA (algo3, B-10k, L.00.33
4.1%
11.7%
0.95
Dataset3-LNN (algo4, B=10k, 0.03)
4.1%
1.6%
0.99
Dataset3-LN (algo5, B 10k, L-0.03)
4.1%
0.4%
0.99
Table 9: False positive and false negative rates for log-normal detection at a threshold of (0.5) alongside with the AUC score. Algorithms are reported in Table [ B stands for the budget used for the attack and L stands for the accepted or distance. LN stands for log-normal.
important and excellent precision is impossible (Sections 4.2 and 4.3 Of course this can not compete with classical local optimization in terms of convergence rates for large budgets, but we show that it can be great for low ratio budget/dimense dimension.
Fake detection presents complex challenge, with the difficulty of identification varying based factors such the type of data and the model used (Epstein et al. 2023 Sha et al. 2023). We consider the setting in (Wang et al. 2023). We observe that log-normal mutations in continuous scttings, as well as other generic black-box optimization algorithms, are credible attack mechanisms, so that detecting attacks is necessary for a good defense. We observe a very poor transfer between the detectors of different types of attacks and in particular from the detection of classical attacks (such as SquareAttack) to our modified attacks (such as log-normal), so that a good defense mechanism (aimed at improving the detection of manipulated images) must include the learning of diverse attack mechanisms.
We also note that detecting no-box attacks is a simple yet effective protection against them: get a detection rate close to 100%, so that no-box attacks are vastly mitigated. Combining (simply by if-then-else) many tools can therefore lead to better fake detectors: classical fake detectors, watermarking detectors, no-box detection as proposed in the present paper, detectors of classical attacks such as SA, and detectors for our proposed black-box atlacks based on generic black-box optimization tools.
Our most immediate further work is the combination, by machine learning instead of if if-then-else, of our fake detectors (no-box, SA, lognormal, and existing detectors such as (Ojha et al. 2023 and (Fernandez et al., 2023). into single detector. We also plan to. add new atlacks and defenses based on generic black-box optimization, and third work is the investigation of log-normal mutations (and other discrete algorithms) for other continuous problems.
13

---

Published in Transactions on Machine Learning Research (02/2025)
Performance of detectors on clean images, images under no-box attacks, and images under black- box attacks. Stable Signature p-value threshold is set to 10-2 The first column corresponds to the TNR (true negative rate), the other colunns correspond to various forms of TPR (true positive rates).
Table 10:
Clean
No-box on watermarked Black-box on Fake
Detectors
Water- marked
GSM Big-LN
Real
Fake
IR
MBT
SA
UnivFD
98.1
99.9
20.8
53.9
24.4
0.0
StableSignature
99.9
0.0
100.0
39.2
45.6
0.5
0.4
29.6
0.4
99.5
0.1
MBT
99.0
1.0
2.2
1.8
100.0
0.1
0.0
SA
98.1
0.0
0.3
0.1
0.0
16.7
100.0
CMA
99.9
8.7
0.0
0.1
0.0
97.6
5.1
Combination
94.6
99.9
100.0
99.8
100.0
99.7
100.0
References
Ali Al-Haj. Combined DWT-DCT digital image watermarking. Journal of computer science, 3(9):740-746)
2007.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: A query-efficient black-box adversarial attack via random search. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (eds.). Computer Vision ECCV 2020 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIII, volume pp. 484-501. Springer, 2020. doi:
12368 of Lecture Notes in Computer Science, 29. URL https: https////oo.org11.100000 org/10..007/
978-3-030-68592-1122
SME Artelys. Sequential quadratic programming: Knitro wins the BBO competition, 2015a. URL https:
www.aartelys.com http://a..oommeeeeeoo/o//eeee///
winner//
SME
Artelys,
2015b.
URL httpss/www.arreelyy..co/////
KNI 1 hctp:pppppppaaaaaaaacoooomaaaaaa Competition
Thomas Bäck and Martin Schütz. Evolution strategics for mixed-integer optimization of optical multilayer systems. In John R. McDonnell, Robert G. Reynolds, and David B. Fogel (eds.), Proceedings of the Fourth Annual Conference Evolutionary Programming, EP 1995, San Diego, CA, USA, March 1-3, 1995, pp. 33 51. A Bradford Book, MIT Press. Cambridge, Massachusetts, 1995.
Thomas Bäck and Martin Schütz. Intelligent mutation rate control in canonical genetic algorithms. In Zbigniew W. Raš and Maciek Michalewiez (eds.), Foundations of Intelligent Systems, pp. 158 167, Berlin, Heidelberg, 1996. Springer Berlin Heidelberg. ISBN 978-3-540-664440-4
Thomas H. W. Bäck, Anna V. Kononova, Bas Stein, Hao Wang, Kirill A. Antonov, Roman T. Kalkreuth, Jacob de Nobel, Diederick Vermetten, Roy de Winter, and Furong Yc. Evolutionary algo- rithms for parameter optimization - thirty years later. Evolutionary Compulation, 31 122, 2023. doi: 10.1162/EVCOO _ URL https: http::://ooiorrroe org/10...100 1ew/1/16eeeee evco 00322
James Bergstra, Brent Komer, Chris Eliasmith, Dan Yamins, and David D Cox. Hyperopt: a Python library for model selection and hyperparameter optimization. Computational Science and Discovery, 8(1):014008, 2015.
Mans-Georg Beyer. The Theory of Evolution Strategies. Natural Computing Series. Springer, Heideberg, 2001.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šindić, Pavel Laskov, Giorgio Gi- acinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Hendrik Blockoel, Kristian Kersting, Siegfried Nijssen, and Filip Zelezný (eds.), Machine Learning and Knowledge Discovery in Databases, pp. 387-402. Berlin, Heidelberg, 2013. Springer Berlin Heidelberg. ISBN 978-3-642-40999-33
14

---

Published in Transactions on Machine Learning Research (02/2025)
Mehdi Boroumand, Mo Chen, and Jessica Fridrich. Deep residual network for steganalysis of digital images. TEFE Transactions on Information Forensics and Securily, 14(5):1181-1198, 2018.
Jianbo Chen, Michael Jordan, and Martin J Wainwright. Hopskipjjmpattaakk: query-efficient decision-
based attack. 2020 iece symposium on security and privacy (sp), pp. 1277-1294. IEEE, 2020.
Davide Alessandro Coccomini, Andrea Esuli, Fabrizio Falchi, Claudio Gennaro, and Giuseppe Amato. De- tecting images generated by diffusers. arXiv preprint arXiv:2303.052: 2023.
Ingemar Cox, Matthew Miller, Jeffrey Bloom, Jessica Fridrich, and Ton Kalker. Digital watermarking and steganography. Morgan kaufmann, 2007.
Swagalam Das, Sankha Subhra Mullick, and Ponnuthurai N. Suganthan. Recent advances in differential evolution an updated survey. Swarm Evol. Comput., 27:1-30, 2016. doi: 10.106/4.SEE00.206..00
URL https://doi org/10 1 swevo 2016. 01.004
Benjamin Doerr and Frank Neumann (eds.). Theory of Ecolutionary Computation Recent Developments in Discrete Optimization. Natural Computing Scries. Springer, 2020. ISBN 978-3-030-29413-7. doi: 10.1007 1110007/9788-3000-9911117
URL htt.://pp///eo..or////000000/000
Benjamin Doerr, Carola Doerr, and Timo Kötzing. Provably optimal self-adjusting step sizes for auult-valued decision variables. In Proceedings of PPSN, volume 9921, pp. 782-791, 09 2016. ISBN 978-3-319-45822-9, doi: 10.1007/978-3-319--2 4582333---6 73.
Benjamin Doerr, Carola Doerr, and Timo Kötzing. Static and self-adjusting mutation strengths for multi- valued decision variables. Algorithmica, 80, 07 2017. doi: 10.1007/5004433-01700031111
Benjamin DoerT, Carola Doerr, and Johannes Lengler. Self-adjusting mutation rates with provably optimal success rules. In Genetic and Evolutionary Computation Conference, pp. 1479 11819-147,411 2019a.
Benjamin Doerr, Christian Gießen, Carsten Witt, and Jing Yang. The (1+A) evolutionary algorithm with self-adjusting mutation rate. Algorithumico, (1(2):593 631, 2019b. doi: 10.107/500433-00-5500-XX URL
10/gg/10..00 10070/00000000 0502-x
Carola Doerr, Hao Wang, Furong Ye, Sander van Riju, and Thomas Bäck. 10Hprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristics. arXiv:1810.0528, 2018.
Carola Doerr, Furong Ye, Naama Horesh, Hao Wang, Ofer M. Shir, and Thomas Bäck. Benchmarking discrete optimization heuristics with iohprofiler. Applied Soft Computing, 186027, 2020. ISSN 1568-4946. doi: https://d6..org/10.10111 201...009..110..0000000.00000000 URL https::/www.sceenceel/ee http://www....eeeeeeeac//aattaaa pis//81568494619080099
Hafsteinn Einarsson, Marcelo Matheus Gauy, Johannes Lengler, Florian Meier, Asier Mujika, Angelika Steger, and Felix Weissenberger. The linear hidden subset problem for the (1+1)-FA with scheduled and adaptive mutation rates. Theoretical Computer Science, 785:150 170, 2019.
Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 382 392, 2023. David C Epstein, Ishan Jain, Oliver Wang, and Richard Zhang. Online detection of ai-generated images. In
Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Toddy Furon. The stable signa- ture: Rooting watermarks in latent diffusion models. arXiv preprinl arXiv:2303.155 15435, 2023.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint Xix: 14:0:111..00 14.265722 2014.
Chuan Guo, Jacob Gardner, Yurong You, Andrew Gordon Wilson and Kilian Weinberger. Simple black- box adversarial attacks. In Kamalika Chaudhuri and Ruslan Salakhutdinov (ods.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 2484-2484 PMLRR 09 15 Jun 2019. URL. https::/prooeeeinngs mlr. S....press/v/) 0t0///guo19e guo.ga.
15

---

Published in Transactions on Machine Learning Research (02/2025)
Ryoki Hamano, Shota Saito, Masahiro Nomura, and Shinichi Shirakawa. CMA-ES with margin: lower- bounding marginal probability for mixed-integer black-box optimization. In Jonathan E. Fieldsend and Markus Wagner (eds.), Proceedings of the Genetic and Evolutionary Computation Conference, Mas- sachusetts, USA, July - 18, 2022, pp. 689-647. ACM, 2022. doi: https://doi.onn org/ 10. 1111/1112/9000--1 35288277
URL
Nikolaus Hansen and Andreas Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, 11))), 2003.
John H. Holland. Adaptation Natural and Artificial Systems. University of Michigan Press, 1975.
Zhengyuan Jiang, Jinghuai Zhang, and Neil Zhenqiang Gong. Evading watermark based detection of ai- generated content. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communica- tions Securily, pp. 1168 1181, 2023.
Johannes W. Kruisselbrink, Rui Li, Edgar Reehuis, Jeroen Eggermont, and Thomas Bäck. On the log- normal self-adaptation of the mutation rate in binary search spaces. In Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, GFCCO '11, pp. 893-900, New York, NY, USA, 2011. Association for Computing Machinery. ISBN 9781450305370. doi: 10.1145/200155
URL
https://doi.oon org/ 1etpg//0..1115... 1ew.11100/0016.... 2001699
T.L Lai and Herbert Robbins. Asymptotically efficient adaptive allocation rules. Adv. Appl. Math., 6(1): 4 2-2 March 1985.
Jialin Liu, Antoine Moreau, Mike Preuss, Jeremy Rapin, Baptiste Roziere, Fabien Teytaud, and Olivier Teytand. Versatile black-box optimization. In Genetic and Evolutionary Computation Conference, pp. 620 628, 2020.
Peter Lorenz, Ricard L Durall, and Janis Keuper. Detecting images generated by deep diffusion models using their local intrinsic dimensionality. In Proceedings of the IEEE/CV /VF/ International Conference on Computer Vision pp. 448 109,-599120 2023.
Laurent Meunier, Herilalaina Rakotoarison, Pak-Kan Wong, Baptiste Rozière, Jérémy Rapin, Olivier Tey- tand, Antoine Moreau, and Carola Doerr. Black-box optimization revisited: Improving algorithm selection wizards through massive benchmarking. IEEE Trans. Evol. Comput., 26(3):490-500, 2022.
David Minnen, Johannes Ballé, and George D Todcrici. Joint autoregressive and hicrarchical priors for learned image compression. Advances in neural information processing systems, 31, 2018.
A. G. Moskowitz, T. Gaona, and J. Peterson. Detecting ai-generated images via CLIP. CoRR, abs/2404.08778,, 2024. doi: 2404. 08788
URL httpp:://oi. 48550/arXiv. 1tt.:/............0////////1.
Ahmad M Nagm, Mohamed Torky, and Khaled Y Youssef. A novel watermarking approach for protecting image integrity based on hybrid sccurity technique. arXiv preprint arXiv:2110.08777 2021.
Weili Nic, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Animashroe Anandkumar. Dif- fusion models for adversarial purification. In International Conference on Machine Learning, pp. 16805 16827. PMLR, 2022.
Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. Towards universal fake image detectors that generalize across generative models. In CVPR, 2023.
Qua Pan, Mehmet Fatih Tasgetiren, and Yun-Chia Liang. A discrete differential evolution algorithm for the permutation flowshop scheduling problem. Computers & Industrial Engineering, 55(4):795-816, 2008. doi:
CWW.:JJJ.EE.000.00..00 URL hnttps://dot org/o0..///o.o.. 1tte//00.006//.11 1110//.cciii. 20.8 03.003
M. J. D. Powell. A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation, pp. 51-67. Springer Netherlands, 1994.
16

---

Published in Transactions on Machine Learning Research (02/2025)
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748 8763. PMLR, 2021.
S. Rahnamayan, H. R. Tizhoosh, and M. M. A. Salama. Quasi-oppositional differential evolution. In JEEE Congress on Bvolutionary Computation, pp. 2229 2236, 2007.
Jeremy Rapin and Olivier Teytaud. Nevergrad - A gradient optimization platform. https://Github. hta/Paacbbokieesaarccclleerrrrge 2018.
Ingo Rechenberg. Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. Frommann- Holzboog, 1973.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the TEEE/CVT conference on computer vision and pattern recognition, pp. 10684 10695, 2022.
Raymond Ros and Nikolaus Hansen. A simple modification in CMA-ES achieving linear time and space complexity. In Parallel Problem Solving from Nature - PPSN X, pp. 296-305. Springer Berlin Heidelberg, 2008.
Mehrdad Saberi, Vinu Sankar Sadasivan, Keivan Rezaei, Aounon Kumar, Atoosa Chegini, Wenxiao Wang, and Soheil Feizi. Robustness of ai-image detectors: Fundamental limits and practical attacks. arXiv preprinl arXiv:2310.000 00076, 2023.
Mehrdad Saberi, Vinu Sankar Sadasivan, Keivan Rezaei, Aounon Kumar, Atoosa Chegini, Wenxiao Wang, and Soheil Feizi. Robustness of ai-image detectors: Fundamental limits and practical attacks. In ICLR, 2024a.
Mehrdad Saberi, Vinu Sankar Sadasivan, Keivan Rezaei, Aounon Kumar, Atoosa Chegini, Wenxiao Wang, and Soheil Feizi. Robustness of ai-image detectors: Fundamental limits and practical attacks. ICLR, 2024b.
Zeyang Sha, Zheng Li, Ning Yu, and Yang Zhang. De-fake:
Detection and attribution of fake images generated by text-to-image generation models. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Securily, pp. 3418 3432, 2023.
Rainer Storn and Kenneth Price. Differential evolution a simple and efficient heuristic for global optimiza- tion over continuous spaces. J. of Global Optimization, (14:33411335 359, 1997.
André Thomaser, Jacob de Nobel, Diederick Vermetten, Furong Ye, Thomas Bäck, and Anna V. Kononova. When to be discrete: Analyzing algorithm performance on discretized continuous problems. In Sara Silva and Luis Paquete (eds.), Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2023, Lisbon, Portugal, July 1 15-19, 2023, pp. 856-863. ACM, 2023. doi: 05555/5588331.3500000 URL
https://doi.org org. 10. 1145/35831111111 3590410
Yizao Wang, Jean- Yves Audibert, and Rémi Munos. Algorithms for infinitely many-armed bandits. In D. Koller, D. Schuurmans, Y. Bengio, and L. Botton (ods.), Advances in Newral Information Processing Systems 21, pp. 1729-1736. Curran Associates, Inc., 2009.
Zhendong Wang, Jianmin Bao, Wengang Zhou, Weilun Wang, Hezhen Hu, Hong Chen, and Houqiang Li. Dire for diffusion-generated image detection. arxiv preprint arXiv:2303.0920 2023.
Thomas Weise, Zijun Wu, and Markus Wagner. An improved generic bet-and-run strategy for speeding up stochastic local scarch. CORR, 088844 2018. URL
Furong Ye, Carola Doerr, and Thomas Bäck. Interpolating local and global search by controlling the variance of standard bit mutation. In IEEE Congress on Evolutionary Computation, CEC 2019, Wellington, New Zealand, June 10-13, 2019, pp. https://dbi.org./ C109//E.....CCCCC 2019. 17198777717
IEEE, 2019. doi: 10.108//EEC22009.5790007 URL
17

---

Published in Transactions on Machine Learning Research (02/2025)
Mingjian Zhu, Hanting Chen, Mouxiao Huang, Wei Li, Hailin Hu, Jie Hu, and Yunhe Wang. Gendet: Towards good generalizations for al-generated image detection. arv preprint or Xix:2312.088800 2023.
A Broader Impact Statement
Detecting fake images is essential for preserving the quality of internet. Our detectors do not provide certainties, only indications. Our attacks could be used against existing detectors: however, our transfer results (Section 6 show that the benefit of our new detector has more impact as it covers other attacks than the ones specifically considered during the training.
B Benchmarks: statistics
Min
Max
Dimension
20000
Budget
10
3000000
Num-objectives
6
Noise dissymetries
False
True
Noise
False
True (many levels)
Number of blocks of variables
16
(with independent rotations)
Number of workers
500
Category
Benchmarks
Real-world, ML. tuning
Keras, Scikit-learn (SVM, Decision Trees, Neural nets)
Real- world, not ML tuning
Crops, rockets, energy, fishing, photomics, game
Discrete
PBO, Bonnans, others (includes: unordered variables)
Table 11: Diversity of the benchmarking platform used in our experiments.
Table presents the benchmarks in Nevergrad.
18

---

Published in Transactions on Machine Learning Research (02/2025)
C
Cases in which normno is outperformed by NGOpt
Cases in which log-normal does not outperform NGOpt
Problem
Rank of LogNormal
Num algorithm
Rank of NGOpt
yaonepenboundelbbobb
7
86
0
yapenboundedbbob
8
87
0
yahdbbob
13
72
7
naivemltuning
14
10
rodueod-yyldllbbbbb
14
yaneeaapenbouudeebbbb
14
88
8
naive-seq-mlltninng
15
23
14
seq-miluning
17
23
8
yabbob
17
81
11
yaboxbbob
17
53
7
18
27
17
instrum-discrete*
23
123
mituning
21
23
9
yaonepenparabbob
22
56
11
double-O-seven
24
39
2
yaboundedbbob
25
89
4
yapenparabbob
31
56
13
yatinybbob
41
89
37
ap-ms-bbob
41
28
ms-bbob
42
65
29
powersystems
42
50
26
yaparabbob
46
61
6
ultrasmall-photonics**
51
85
28
mldakmeans
57
70
13
Table 12: Benchmarks which log-normal performs weaker than NGOpt. NGOpt wixard, automatically choosing algorithm in big portfolio of algorithms: it performs vastly better than log-normal for problems with noise (007 and PowerSystems, for which noise management is essential: log-normal can do better for this when combined with Optimism in Front of Uncertainty for dealing with the noise as detailed in Section D.1 and Fig. 2), and for problems derived from BBOB in which high precision by continuous methods is possible.
Black-box Optimization Algorithms
D.1 List of methods
We briefly present the main BBO algorithms used in the present paper.
Log-normal (the full name in Nevergrad is LogNVrrnaDissrreeenePPisse
sometimes abbreviated
as LDOPO), (1 + A) EA optimizer embedded with the self-adaptive log-normal mutation. This
optimizer is adapted from the corresponding discrete version (Kruisselbrink et al. 2011] (see Sec-
tion 3.1). and details of the tested continuous optimizer are introduced in Section 3.2 abbreviated as T.DOPO.
sometimes
Lengler (DiscrreteLeegleeOnneee OncellerreePlluooooe sometimes abbreviated as Lglr), namely the scheduled muta- tion rate in (Einarsson al. 2019).
Adaptive mutation rates (AdpttiaDisrretCnhPhhhonnn come from Doerr al. 2019a). This is a (1 1) EA with self-adjusting mutation rates. Tt. increases the mutation rate by F p when the
19

---

Published in Transactions on Machine Learning Research (02/2025)
obtained offspring is at least good as the parent. Otherwise, p is replaced by p/F. F and are two constant parameters. The algorithm is extended to the continuous optimization following the strategy in Section 3.2
Anisotropic discrete algorithms, which uses self-adaptation with mutation rate per variable.
NGOpt is a wizard developed by Nevergrad (Rapin Teytaud 2018 antomatically chooses an algorithm from portfolio of algorithms.
CMA-ES, the Covariance Matrix Adaptation evolution strategy, is well-known continuous op- timization algorithm proposed by Hansen (Hansen & Ostermeier 2000)). We consider CMA and Dia (Ros & Hansen, 2008) implemented by Nevergrad (Rapin & Teytaud 2018 for com- parison: the latter is modified version for high dimensional objective functions.
Random Search (abbreviated RS) has been commonly applied as a baseline for algorithm comparison. In this paper, we sample new values for each variable uniformly random.
MuluSQQ, a combination of multiple sequential quadratic programming runs (Artelys) 2015b Rapin
Teytaud 2018
CMandAS2, combination of several optimization methods (depending on the dimension and bud- gel, e.g. several CMAs equipped with quadratic MetaModels combined in bet-and-run Weise el al. 2018) or the simple (1 11+1 evolution strategy with one-fifth rule mneeheeceen
1973
implemented
(Rapin & Teytaud 2018
NGOptRW, the wizard proposed for real-world problems in Rapin & Teytaud 2018 DE and more PSO than NGOpt (as opposed to using CMA), and a bet-and-run Weise et al. 2018
used more
Carola2, a chaining (inspired by (Deerr et al. 2018) between Cobyla (Powell 1994). CMA Hansen
& Ostermeier 2003 accompanied by meta-mode model and sequential quadratic programming Artelys 2015ab.
HyperOpt (Bergstra et al. 2015). based on Parzen estimates.
Ngloh variants, recent wizards co-developed by the Nevergrad team and the IOH team (Doerr al. 2018).
CMA variants: besides DiagonalCMA (Ros Hansen 2008 we consider LargeDiagonalCMA (as DiagonalCMA, but with larger initial variance for the population).
VastDE, basically DE (Storn & Price 1997 sampling closer to the boundary in bounded cases or with greater variance unbounded contexts.
SQOPSO, Special Quasi Caais-Oppsstte PSO, which adapts quasi-opposite sampling (Rahnamayan et al. 2007) to PSO.
BAR4, bet-and-run between
quasi-opposite DE folowed by BFGS with finite differences [Rahnamayan et al. 2007) and
CMA equipped with meta-model followed by sequential quadratic programming part.
Other methods were run thanks to their availability in the Nevergrad framework; we refer to Rapin & Teytaud 2018) for all details.
We use the terminology in the Nevergrad code, i.c., (1+ optimization methods are derived from the (1+1) code, and contain the sulfix OnePlusOne even if X 1.
20

---

Published in Transactions on Machine Learning Research (02/2025)
Parameter
Value
Frequency of update
1/2 (ZelaSmooth)
(per iteration)
1/3 (Ultrammooh))
1/9 (SuperSmooth)
1/55 (Default smooth)
Size of smoothing window s
Table 13: Parametrization of the Smooth operator, which operates on a s-window.
D.2
Modifiers of algorithms
D.2.1 Modifiers dedicated to tensors
When the Smooth operator is applied to black-box optimization method in Nevergrad, periodically, it tries to replace the current best candidate x by Smooth(z). If Smooth(x) has better loss value than I, then x is replaced by Smooth(r). Smooth(r) is defined as a tensor with the same shape as with Smooth(()) defined as 2ς with probability 75%, and, otherwise, the average of the Iy for j the points at distance at most of the indices of the tensor æ. Smooth means that this tentative smoothing tested once per 55 iterations, SuperSmooth once per 9 iterations; see Table
D.2.2 Modified dedicated to adversarial attacks
The modifiers and GSM used for arrays work as follows:
lossg(x) loss(0.03 sign(z))
lossasM (I) loss(0.03 sign(comuolve((z k)))
in the context of an image with values in [0, and an amplitude 0.03 in 1 k refers to Gaussian kernel of width r/8 with the width of the image.
D.3 Other modifiers of algorithms proposed in Nevergrad
There are other modifiers of algorithms proposed in Nevergrad and visible in prefixes, as follows:
The prefix SA (self-avoiding, referring to tabu lists) and the suffix Exp (referring to parameters related to simulated anmealing) refer to add-ons for the discrete optimization methods.
Recombining (R for short), which adds two-point crossover (Holland 1975 in algorithm.
Acceleration by meta-models: by default, MetaModel means CMA plus quadratic meta-model. but an optimization method (DE, PSO or other) can be specified, and the meta-model can be a random-forest (RF) or a support vector machine (SVM) or a neural network (NN). The algorithm periodically learns a meta-model, and if the learning looks successful it uses the minimum of the MetaModel as a new candidate point.
Combination with Optimism in front of uncertainly: Nevergrad features bandit tools, which can be combined with other algorithms for making them compatible with noisy optimization. For example, the optimistic counterpart of an algorithm A performs an upper confidence bound mea & Robbins 1985) for choosing, between points in the search space that have already been used, which one should be resampled, and chooses a new point when the number of function evaluations exceeds a given function of the number of distinct points as specified in (Wang et al. 2009 Rapin & Teytaud 2018)
E Significance in Nevergrad plots
In figures created by Nevergrad all dots are independently created. This means, for example,that dots obtained for budget 1000 are not extracted from truncations of runs obtained for budget 2000. This implies
21

---

Published in Transactions on Machine Learning Research (02/2025)
that the robustness of the rankings between methods can be deduced from the stability of curves: if the curve corresponding to algorithm A is always below the curve obtained for Algorithm B (in minimization) this means that Algorithm A performs robustly better than Algorithm B. For computing p-values, the probability that Algorithm performs better than Algorithm B for the & greatest budget values at most 1/2* under the null hesos that they have the same distribution of average loss values. When computing ranks of algorithms as in Tables and 12 Nevergrad proceeds as follows:
Compute the average obtained loss lossa.p.b for each algorithm and each problem and each budget b (averaged over instances).
Then the score of a w.r.t algorithm d' is scorea,''s frequency at which lossa, < lossa' p..b
Then, given n the number of algorithms, the score of a is scorea 1 2 Ee scorea,a' and Nevergrad provides the rank for this score.
F Comparisons of log-normal variants
Figures and compare Lognormal variants on many problems. The optimistic variant is unsurprisingly good for noisy problems and topological problems seem benefit from anisotropic mutations. Besides that, the standard Lognormal version is never very far from the optimum.
22

---

Published in Transactions on Machine Learning Research (02/2025)
double p. crwwn, plwcc1 Nigeel
1.12
1x10
LOCAL ling plantl akpoc
CMARGU
4-127
Fandenttr of IL
1+17
1-12
X 10/022
veluuing p00011 MgOo
2513
W
Figure 2: Analysis of variants of Lognormal mutations (1/2). The number between parentheses is the average score for the maximum budget and the number between brackets is the average score for the penultimate budget: as these two figures are obtained in completely independent runs the consistency between both shows the obbustues//sinniiiaane variants (Section D.1b of log-normal algorithms perform well for noisy optimization problems such as 007.
of the ranking (more details in Appendix Unsurprisingly, the Optimistic
23

---

Published in Transactions on Machine Learning Research (02/2025)
agaa
10022020 2 stul Aggrt
1.
23
yorrs Beek parast WEA
spoless reteaiates pREatS algoo
1.11
4.15
3x
2x13
Figure 3: Analysis of variants of Lognormal mutations (2/2). We observe that the default parametrization of LogNormal is essentially ok. Lognormal mulations are serious competitor in some continuous problems when the prior search distribution used implicitly initialization and mutation is good and the dimension is high (e.g., YAHDBBOB). For topology optimization, the variable-wise adaptive mutation rate Anisotrop- ic PexdapttieDiisrreeeeeelnnneeeee (see Section D.1 is excellent.
24

---

Published in Transactions on Machine Learning Research (02/2025)
G
Parameters of the learning model
Hyperparameter
Value
Type of model
SRnet
learning rate
10
Number of epochs
20
Hardware
1 GPU
Training time
= hour
JPEG 0.2 prob at 96 quality Resize((256, 256))
Data Augmentation
RandommHorizoottallll(((..... (andomCroop(224, 224))
Table 14: Hyper-paramcters of our learning model. We work on Dataset2 with cach purifier (Datasct2-DP and Dataset2-IR) with parameter 0.1, which split in 80% train, 10% Lest, 10% validation.
H Full details of the preliminary experiments: LogNormal performs well on Nevergrad benchmarks
25

---

Published in Transactions on Machine Learning Research (02/2025)
instrum discrete plots( 14 algos)
8x
wus.poowcccapppoooooxoeeeeeeeeee (0. (0.746 <0.746>))
NaD533rdered (0.441 (0.44100.611)
No NalhhLnDrdered [0 429 <0-5
(0.427
NgLnnrrdeeed [0.424 00.0000>1
IcJohhnunnoden [0.4] 601>)
NoD 10.edd00.430 +0.5505533
X5m Lognooom Discettonnseuuuueeeeeee
(0:411 <0.551>>
5x
-6- Ni NglonnSunordor (0 40 <0 (0/5))))
-9
Nilhhorrerje 10.404 5322
392s]
NciohhunedDrdee (0.403 <051>>)
Naighz (rrdrred
<0.53
.8.
Iloon (0,402 5333>>
-- No Naich unnee (0 (0.402 44400x200 <0.532>1
10-1
XSmeillognomma wow.ooooooaaalaannooo
NguSZOrdered (0 201 <0.3552
567>)
(0 50.52
BigLognorma hiammaiisseeeeeeeeuuueee 10 10:79 <0.5385)
Sma Legnemm Srrrecnennnrrrreeeeeeeee
(18=)
10a
10
<0.50>>
CDDsRiUnnareere (0,374 <0.422))
budget
NaRSordesedd (0,372 <0446-))
usOneordered 00.
Discretel Eescreeeengieer Lengierrmmee eeteerroeasssnnneeeerrre (0.37) <0.496))
Nagssn (0:31 <0.512>)
10,37
NsspptUrdered 10.37 <0.07>)
XLoghorma Disc eetenneliggneeunrrreeee (0 583>)
<0.51 Diccett@nnekudrnerrrre
XLoanorma SetceeeetooPPsseeeee
(0.367 <0.588>)
Lognormal hww.a//DDDseeetteeeesooonnnnonoo
(0.365 <0.51>>)
abo_reducedd cecuuccdduutee_pee Suite_plots(3 algos)
hilpsmmoileddiiillsssreeeeeeeees 00. <0.811>>)
MGDSRWorderee) (0.113.40.164
(0.112 <0.163>)
[0.112 <0.16>)
MGOpttnordenss
[0:111<D::61]]
DiscleteLengle: www.eengggeeePPPssSeeeeee oneeiioeeeesnneeeeueddeeeeeeeee (0:11 <0.162>)
loss
(0.167/)
S0OPSCD (0.11<0.222.
NoLolrorderez
(0 (0.10)<0.1155>)
LPC BPCMAUnnrder 00,107 <0,185>)
tat 001107 <0 2645|
NGOprSottmax (1 105 <0273>1
.96 X5mall Darrrmmmaeeeeeeeelll..
(0:105 <0.208>1
NGRSOrderee (0. (0.103 103 <0 153>]
10-1
NGDSAW inddered (0 102 27.172>
NoDS 30rdeed (0 0g55 2161
Lnodened
<0.2121
NGloo Unordered (0 2946 <0
NaDS3 Unarderdd (0 (0.044 <0.2167)
NGLnnnooreeee www.ereeeeeeeeeenneeeeee
10a
10a
10a
XSmaal Lognornal
(0 091 <0.178>) -0.131_,01177000
Lognem abbiscretsOhee SheeerrhePPusOnee
(0.0855 Seeeluugnneeooodeeeeeedd (0.09 <0.13>>)
budget
"LoghomaalDisco BeccettOmeeluuonnee
(0 0878 <t 165>))
Lognormal Dissrmaiiiirrreennee
(0. 0358 <0 16.1)8>)
CMA nordered (0.0788 ed.1612)
Hugel ognommllDisrreee Disereiiiieeeemmmm
Susoononnnnnnmmeee 00.070 <0 207>)>
00.0701 <0 2072)
Biggonnamal
(0. 0645 00.452007777 20.17>]
Figure 4: Results on instrum-discrete (top, 35 best methods and the worst method out of 98 methods run
on this benchmark), pbo-roduced (bottom, 35 best and the worst method out of 156 methods run on this benchnark). log-normal simple but good. CMALn is a combination of CMA and log-normal (used as a warmup during the early 10% of the budget): on PBO all strong methods use log-normal at some point.
26

---

Published in Transactions on Machine Learning Research (02/2025)
Benueealiiinoonissssrrreeeeeeeoe (0.503 <0.519>)
USbesi
LC 40
deceptive (38 algos)
Nalve (0.342 (0.34 <0.367>)
DE 10 34133 34121
10.297 <0
LODDE (0.297 <0
Kandomsearch 10.294 <0,312>>
Naioh1 7 258 25/2)>>
CRA (0.257 20.77>)
NODS (U 252 c0.3025)
to 251 <0.255>)
NGDSKW
CSECS (022 c0,2275)
NGOpt
215 <0
NO 20.12 20,212)
l0ss
C0 2022821
<0,21=>))
.9. CSE
Ce 209 532275)
CHARS 0.207 202313
CS 10 <0 2622
CS 10 20 20
NoRs (0
CRALD 1o 10 <020>>)
NgLn 14 20.19
Huge www.ccooomaaDiseeeeooooooo
(0,0751 <0.0834>)
Loghorma ScrermaiDDseeeeeeeeeussoooo (0.0591 <0.106>)
Discretelenggee@@@eeuuoeee (0.0449 <0.0848>)
10a
10a
10*
budget
Figure 5: Results on the Deceptive benchmark in Nevergrad. X-axis: budget. Y-axis: average normalized (linearly to [0, 1]. for each benchmark) loss. We observe that CMALn (CMA with log-normal warmup) out- performs CMARS (CMA with random search warmup), which outperforms CMA, on this hard benchmark. The best algorithms are based on log-normal: CMALn (resp. NgLn) uses CMA (resp. NGOpt) for local op- timization after log-normal. Lengler also performs well, showing that discrete algorithms can be competitive warmup
for continuous problems, in the hardest cases, are all variants of Ngloh Tuned: they are good, but still outperformed by codes based Lognormal. We note an excellent performance of the Lengler method adapted to continuous problems, in particular for the greatest values of the budget, though Table 2 shows that log-normal was better over the different budget values (25, 37, 50, 75, 87, 100, 200, 400, 800, 1600, 3200, 6400, 12800) for the criterion defined in Section
standalone method. The CSEC codes
27


Reviews:
[{"review_id": "JOn3EYGZgF", "reviewer": "Reviewer_wxv9", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**:\n* Detection of fake images and robustness of fake image detectors is an important topic\n* I have faith in validity of experimental results (i.e., I see no red flags and inconsistencies)\n* It is a good demonstration that detectors trained to detect a specific attack can fail in detection of other attack types\n* Authors show that their attack is detectable (one can train a detector) even if parameters of the attack used during training are different from those during testing.\n\n**Weaknesses**:\n* In adversarial machine learning, we typically only care about imperceptible attacks. SQUARE attack is such an attack (L_inf with eps = 0.03). Is the proposed attack in the same category? I am not sure. In section 5, you basically provide a definition of an L_inf attack without naming it (so please name it explicitly). However, in algorithm 2, line 6, you seemingly sample a pixel \"replacement\" from the full domain (i.e., all pixel values).\n* Given that there can be potentially an infinite number of various attacks, these unseen attacks can potentially evade detection by existing detectors. Once a new attack becomes known, it may (or may not) be possible to train a detector for this attack and expand a list of detectors. But what is an end game for this? I do not expect the paper to solve this problem (possibly unsolvable), but I found no discussion of this issue.\n* Although this is not a dealbreaker, the paper is somewhat hard to read. For example, there is no simple introduction/explanation of what is a fake image (and fake detector). I found no definition of no-box attacks. There is no discussion of what is an imperceptible adversarial attack.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes to apply log-normal mutations to modify fake images (potentially generated by AI image generators). Authors find that these attacks avoid detection by several standard detectors, in particular, these trained to detect a SQUARE attack. Then they show it can be, in turn, mitigated by training detectors on long-normal attack data.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* Please, introduce the issue of fake images and adversarial attacks on fake image detectors.\n\n* Briefly describe the issue of imperceptible adversarial attacks (listing some common types of attacks on images would be beneficial too).\n\n* Please, clarify whether you use only L_inf attacks. In section 5, you basically provide a definition of an L_inf attack without naming it (so please name it explicitly). However, in algorithm 2, line 6, you seemingly sample a pixel \"replacement\" from the full domain (i.e., all pixel values).\n\n* What is the exactly the population size lambda? Which population are we talking about here?\n\n* In the end of the paper you write about plans to expand your toolbox with new attacks and defenses. Does it solve all the problems? Please, **shortly** discuss a potential issue of new attacks being invented.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bFvfJ0ndiF", "reviewer": "Reviewer_Ngu7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\nExtending log-normal mutations from discrete to mixed and continuous domains is an innovative technical contribution.\n\nThe paper presents a thorough comparison of log-normal mutations against other optimization algorithms using the Nevergrad benchmarks and performs extensive experiments.\n\n**Weaknesses**\n\nI must say I had a particularly hard time trying to read/understand this manuscript. The work makes strong assumptions about the knowledge of readers about evolutionary algorithms and the experimental suite that's being extensively used. TMLR is not a venue that specifically publishes research on evolutionary algorithms (EAs) and furthermore evolutionary algorithms are not the mainstream line of research for neither adversarial attacks nor defenses. The authors should make a clearer introduction on EAs specifically on this problem and perhaps briefly explain how EAs differ from gradient-based approaches which has been the dominant line of research for adversarial examples. It would also be great if there was a gentle introduction to Nevergrad and perhaps provide some examples for the tested benchmarks. Are these benchmarks even relevant for the adversarial attack/defense community? This is the first time I'm coming across these benchmarks in this context. What are some prior works that use this benchmark in this context and why do they matter? \n\nFigure 1, Figure 2, and Figure 3 are impossible to read. I suggest the authors to provide two versions of these Figures where the first one has every optimization method grayed out except for the proposed one and the second one has everything with colors/icons.\n\nImage datasets used in the paper (provided in Table 3) are too convoluted. I cannot follow the logic of the authors in introducing 11 datasets while only a handful is mentioned/used. I'm comparing this work to the work of [1] which introduces the Square Attack (SA), which is the primary benchmark used in Table 4, and it would be sufficient to simply use Imagenet-1k similar to how it is used in [1]. A simple table such as the Table 2 in [1] which demonstrates success/failure rates on a subset of Imagenet with several models is all that's necessary. I also cannot understand how the authors come to the conclusion provided in the caption of Table 3, which states \"The key point for us is that it (the proposed method) performs reasonably well while the transfer from SA detectors fails\". SA is denoted to have 100% success rate. How does it fail?\n\nI assume the reason the authors used all other optimization methods for Nevergrad benchmarks and not on ImageNet is because of the automated nature of Nevergrad benchmarking suite. How do (some of) those methods perform on ImageNet compared to the proposed method?\n\nWhat is the average and the median query necessary for success? The authors limit total queries to 10,000 but that does not mean that all attacks are going to use that many queries. For example, SA also uses 10,000 query limit in their work but average used query (depending on the model) is usually less than 1k.\n\n[1] Andriushchenko et al. Square Attack: a query-efficient black-box adversarial attack via random search", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The work explores the application of log-normal mutations, a concept originating from evolutionary algorithms, for adversarial attacks and defenses for fake image/data detection. The authors extend the log-normal mutation method to continuous domains and demonstrate its effectiveness in creating adversarial examples that evade detection by existing fake image detectors. Furthermore, the paper introduces new detectors designed to identify these novel attacks, integrating them with fake detection systems to enhance robustness. Extensive benchmarking on the Nevergrad platform and experiments in fake image detection highlight the utility of the proposed method in diverse scenarios.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "- Justify why Nevergrad benchmarks are relevant in the context of adversarial attacks/defenses when the majority of the research is conducted in image domain.\n- Simplify experimental settings that involve ImageNet and employ a setting similar to [1].\n- Report average and median necessary queries.\n- Compare the proposed method to SA in low-query regimes.\n- Compare the proposed method to some of the other EA methods in image domain.\n- Demonstrate experiments with $L_2$ constraints.\n- Rewrite the parts of the manuscript where the reader is expected to have large amount of prior knowledge on EAs (in particular, Section 4).", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "QIgoFZnhKI", "reviewer": "Reviewer_6yDZ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n1. The proposed algorithm is clearly explained and easy to follow. Moreover, it is easy to implement.\n\n2. The problem of identifying and understanding adversarial attacks is critial.\n\n\n **Major Concerns:**\n\n1. Effectiveness. Table 4 shows that the baseline method SA achieves a 100\\% success rate, which is clearly better than the proposed algorithm, even when the hyperparameters of the proposed algorithm are fine-tuned. It raises my concern how effective the proposed method is. In particular,\n   - Why bother using the propose algorithm if an existing method does a better job? \n   - While the authors show a list of datasets used in the study, only results on the dataset 1 is reported. Where are the results on other datasets?\n   - How to select hyperparameters in an efficient way in practice? \n   - Is 10k query budget for each adversarial image or for the whole dataset?\n2. Lack of comparisons. There are a number of black-box adversarial attacks developed in the literature besides SA. The authors should add comparison to more baseline methods. Also, the authors should also discuss them as related work. \n3. Experiment results. Table 6, 7, 8, 9 has a surprising pattern that the FPR for multiple settings are exactly the same. Can authors explain the reasoning behind? \n4. The authors spend a great amount of energy explaining the benefit of choosing log-normal as black-box optimization algorithm. Nevertheless, quoted, \"We observe that the log-normal algorithm can outperform NGOpt on 17 out of 41 benchmarks\", then why not use NGOpt? Is there any evidence that the deceptive benchmark is strongly connected to the task of creating an adversarial example, or optimizing over the adversarial objective?\n5. Contributions. The authors state that they generalize log-normal mutation from discrete case to the continuous case, while in Section 3.2, the authors also acknowledge that \"This technique has also been adopted by Nevergrad (Rapin & Teytaud, 2018)\". Can authors clarify their contribution on this matter? \n\n**Minor Concerns:**\n\n1. I feel that the introduction part highlights too much on the black-box optimization, while most of them are unrelated to the main content. On the other hand, the introduction and related work on fake image detection is largely ignored. Similarly, Section 2 emphasizes fake detectors but this paper mostly focuses on designing attacks. \n2. Figure 1 (and others) is somewhat difficult for the reader to extract useful information since there are too many lines.\n3. The proposed defense algorithm is straightforward, namely training a detector on log-normal attacked images.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a black-box adversarial attack to evade fake image detector. Given the access to the probability scores predicted by the detector model, the proposed evolutionary algorithm with log-normal mutation explores potential adversarial input by randomly mutating the value of some coordinates of the input. Experiments show that this attack can produce images that successfully fool a pre-trained fake image detector and defenses designed for other attacks such as denoising and purification. The authors therefore propose to defense the proposed algorithm by training an additional detector for log-normal attacks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see concerns above.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: There were two key concerns: technical soundness and clarity. Both have been adddressed to the reviewers' satisfication.

1. Technical Soundness. The experimental results were validated by multiple reviewers, with updates such as clarifying Linf norm constraints and detailing the log-normal mutation’s mechanics enhancing rigor.

2 Clarity and Accessibility. Definitions of key terms, a restructured introduction, and simplified experimental settings addressed Reviewer Ngu7’s concerns about accessibility. Moving ancillary benchmarking results to the appendix has simpliefied the main paper.

Relevance and Broader Implications: The authors demonstrated their approach onreal-world adversarial detection scenarios while acknowledging limitations. The inclusion of new detectors trained on generic black-box attacks provides transferable insights for detecting novel attack methods.

In summary, the paper makes a clear contribution to adversarial robustness in fake detection, and its interdisciplinary approach broadens the scope of tools available for addressing this important problem.
Decision: Accept (poster)