# The percolating cluster is invisible to image recognition with deep learning

Djénabou Bayo, Andreas Honecker, Rudolf Römer

► To cite this version:

Djénabou Bayo, Andreas Honecker, Rudolf Römer. The percolating cluster is invisible to image recognition with deep learning. New Journal of Physics, 2023, 25 (11), pp.113041. 10.1088/1367-2630/ad0525. hal-04054794

HAL Id: hal-04054794

https://hal.science/hal-04054794v1

Submitted on 3 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
PAPER • OPEN ACCESS

# The percolating cluster is invisible to image recognition with deep learning

To cite this article: Djénabou Bayo et al 2023 *New J. Phys.* **25** 113041

View the [article online](#) for updates and enhancements.

You may also like

- [Cluster structure and dynamics in gels and glasses](#)
R Pastore, A de Candia, A Fierro et al.

- [How soon after a zero-temperature quench is the fate of the Ising model sealed?](#)
T. Blanchard, F. Corberi, L. F. Cugliandolo et al.

- [Coarsening and percolation in the kinetic 2d Ising model with spin exchange updates and the voter model](#)
Alessandro Tartaglia, Leticia F Cugliandolo and Marco Picco
<figure><img src="image_2.png" /></figure>

CrossMark

OPEN ACCESS

RECEIVED

17 April 2023

REVISED

18 September 2023

ACCEPTED FOR PUBLICATION

19 October 2023

PUBLISHED

23 November 2023

Original Content from
this work may be used
under the terms of the
Creative Commons
Attribution 4.0 licence.

Any further distribution
of this work must
maintain attribution to
the author(s) and the title
of the work, journal
citation and DOI.

<figure><img src="image_3.png" /></figure>

PAPER

# The percolating cluster is invisible to image recognition with deep learning

Djénabou Bayo¹,²,* ID, Andreas Honecker² ID and Rudolf A Römer¹ ID

¹ Department of Physics, University of Warwick, Coventry CV4 7AL, United Kingdom

² Laboratoire de Physique Théorique et Modélisation, CNRS UMR 8089, CY Cergy Paris Université, Cergy-Pontoise, France

* Author to whom any correspondence should be addressed.

E-mail: Djenabou.Bayo@warwick.ac.uk

**Keywords:** percolation, machine learning, convolutional neural networks, residual learning, correlation function, spanning cluster

## Abstract

We study the two-dimensional site-percolation model on a square lattice. In this paradigmatic model, sites are randomly occupied with probability *p*; a second-order phase transition from a non-percolating to a fully percolating phase appears at occupation density *p*<sub>c</sub>, called percolation threshold. Through supervised deep learning approaches like classification and regression, we show that standard convolutional neural networks (CNNs), known to work well in similar image recognition tasks, can identify *p*<sub>c</sub> and indeed classify the states of a percolation lattice according to their *p* content or predict their *p* value via regression. When using instead of *p* the spatial cluster correlation length ξ as labels, the recognition is beginning to falter. Finally, we show that the same network struggles to detect the presence of a spanning cluster. Rather, predictive power seems lost and the absence or presence of a *global* spanning cluster is not noticed by a CNN with a *local* convolutional kernel. Since the existence of such a spanning cluster is at the heart of the percolation problem, our results suggest that CNNs require careful application when used in physics, particularly when encountering less-explored situations.

## 1. Introduction

Convolution neural networks (CNN) are a class of deep, i.e. multi-layered, neural networks (DNNs) in which spatial locality of data values is retained during training in a machine learning (ML) setting. When coupled with a form of residual learning [1], the resulting residual networks (RESNETS) have been shown to allow astonishing precision when classifying images, e.g. of animals [2] and handwritten characters [3], or when predicting numerical values, e.g. of market prices [4]. In recent years, we also witnessed the emergence of DNN techniques in several fields of physics as a new tool for data analysis [5–8]. In condensed matter physics in particular, DNN and CNN proved to be performing well in identifying and classifying the phases of matter or learning order parameters [9–14]. ML has also been shown to be efficient in learning the ground state of some many-body system [15–19], and to speed up Monte Carlo sampling [20–22].

Despite all these studies, the ML process in itself tends to be somewhat of a black box, and it is not yet known what is allowing a DNN to correctly identify a phase. To gain further insight into this issue, we choose a well-known and well-studied system that exhibits perhaps the simplest of all second-order phase transitions, the *site-percolation* model in two spatial dimensions [23, 24]. In this model, a cluster spanning throughout the system emerges at an occupation probability *p*<sub>c</sub>, leading to a non-spanning phase when *p* < *p*<sub>c</sub> while *p* > *p*<sub>c</sub> corresponds to the phase with at least one such spanning cluster [24]. Several ML studies on the percolation model have been already published, mostly using *supervised* learning in order to identify the two phases via ML classification [25, 26]. An estimate of the critical exponent, *ν*, of the percolation transition has also been given [25]. The task of determining *p*<sub>c</sub> was further used to evaluate different ML regression techniques in [27]. For *unsupervised* and *generative* learning, less work has been done [25, 26, 28]. While some successes have been reported [28, 29], other works show the complexities involved when trying to predict percolation states [26].
In this work, we start by replicating some of the supervised deep learning (DL) analyses. We find that CNNs usually employed in image recognition ML tasks also work very well for classifying percolation states according to *p* as well as for regression when determining *p* from such states. The results are less convincing when instead of *p*, we use the spatial correlation lengths ξ as an alternative means to characterize the phase boundary. We find that, even when correcting for probable difficulties due to non-balanced data availability for ξ, classification and regression tasks fail to give acceptably diagonal confusion matrices. Crucially, when analyzing in detail whether spanning clusters < *p*<sub>c</sub> or non-spanning clusters > *p*<sub>c</sub> are correctly identified, we find the CNNs that performed so well in the initial image recognition tasks now consistently fail to reflect the ground truth. Rather, it appears that the CNNs use *p* as a proxy measure to inform their classification predictions—a strategy that is obviously false for the percolation problem. We confirm this conclusion by testing our networks with bespoke test sets that include artificially spanning clusters < *p*<sub>c</sub> or firebreak structures for > *p*<sub>c</sub>.

## 2. Model and methods

### 2.1. The percolation model

The percolation problem is well-known with a rich history across the natural sciences [23, 24, 30–33]. It provides the usual statistical characteristics across a second-order transition such as, e.g. critical exponents, finite-size scaling, renormalization and universality [24]. Briefly, on a percolation lattice of size *L* × *L*, individual lattice sites $\vec{x} = (x, y)$, $x, y \in [1, L]$, are randomly occupied with occupation probability *p* such that the state $\psi$ of site $\vec{x}$ is $\psi(\vec{x}) = 1$ for occupied and $\psi(\vec{x}) = 0$ for unoccupied sites. We say that a connection between neighboring sites exists when these are side-to-side nearest-neighbors on the square lattice, while diagonal sites can never be connected. A group of these connected occupied sites is called a *cluster*. Such a cluster then *percolates* when it spans the whole lattice either vertically from the top of the square to the bottom or, equivalently, horizontally from the left to the right. Obviously, for *p* = 0, all sites are unoccupied and no spanning cluster can exist while for *p* = 1 the spanning cluster trivially extends throughout the lattice. In figure 1, we show examples of percolation clusters generated for various *p* values. The *percolation threshold* is at *p* = *p*<sub>c</sub>(*L*), such that for *p* < *p*<sub>c</sub>(*L*) most clusters do not span while for *p* > *p*<sub>c</sub>(*L*) they do. This can be expressed as the *percolation probability* $\mathcal{P}(*p*) = \langle s_L(*p*)/L^2 \rangle$, where $s_L(*p*)$ gives the size of the (largest) percolating cluster for size *L* and $\langle \cdot \rangle$ denotes an average over many randomly generated realizations. Similarly, we can define the probability of non-percolating, $\mathcal{Q}(*p*) = \langle (L^2 - s_L(*p*))/L^2 \rangle$ and $\mathcal{P}(*p*) + \mathcal{Q}(*p*) = 1$. We emphasize that $\mathcal{P}$ and $\mathcal{P}$, with the latter shown in figure 1(a), are different. The parameter $\mathcal{P}$ corresponds to the probability of a site to belong to the largest cluster and converges to the *order parameter* $\mathcal{P}_\infty$ when $L \to \infty$. The quantities $P(*p)$, $\mathcal{Q}(*p)$ denote the probabilities of presence or absence of the spanning cluster at a given $p$, respectively<fnref n="3" />. For an infinite system ($L \to \infty$), one finds the emergence of an infinite spanning cluster at $p_c = 0.59274605079210(2)$. This estimate has been determined numerically ever more precisely over the preceding decades [34] while no analytical value is yet known [33]. Another quantity often used to characterize the percolation transition [24] is the two-site connected cluster correlation function, $g(r) = \langle \psi(\vec{x})\psi(\vec{x}+\vec{r}) \rangle_{\vec{x},\vec{x}+\vec{r}\in C, r=|\vec{r}|} - \mathcal{P}^2$, where the $\langle \cdot \rangle_{\vec{x},\vec{x}+\vec{r}\in C, r=|\vec{r}|}$ denotes the average over all $\vec{x}$, all distances *r* and all clusters {C} with $\vec{x}$ and $\vec{x}+\vec{r}$ belonging to the same cluster $\mathcal{C}^4$. This $g(r)$ hence measures the probability to have two occupied sites in the same non-spanning cluster separated by a distance *r*. The associated correlation length ξ is determined through $\xi = \sqrt{\sum_r r^2 g(r) / \sum_r g(r)}$. In the infinite system ξ diverges in the presence of spanning clusters at $p_c$ as $|p - p_c|^{-\nu}$, where $\nu = 4/3$ is the critical exponent, determining the universality class of the percolation problem [24]. We shall use $\langle \xi \rangle$ to denote the average over the sample configurations.

### 2.2. Generating percolation states for training and validation

In order to facilitate the recognition of percolation with image recognition tools of ML, we have generated finite-sized *L* × *L*, with *L* = 100, percolation states, denoted as $\psi_i(*p)$, for the 31 *p*-values 0.1, 0.2, ..., 0.5, 0.55, 0.555, 0.556, ..., 0.655, 0.66, 0.7, ..., 0.9. For each such *p*, *N* = 10000 different random $\psi_i(*p)$ have been generated. Each state $\psi_i(*p)$, $i = 1, ..., N$, is of course just an array of numbers with 0 denoting unoccupied and 1 occupied sites. Nevertheless, we occasionally use for convenience the term ‘image’ to denote $\psi_i(*p)$. The well-known Hoshen-Kopelman algorithm [35] is employed to identify and label clusters from which we (i) compute $s(*p)$, $g(*p)$, and $\xi(*p)$ as well as (ii) determine the presence or absence of a spanning cluster. In figure 1 we show examples of percolation states generated for various *p* values as well as

<footnote n="3">³ In the notation of [33], $\mathcal{P}$ and $\mathcal{P}$ are finite-$L$ versions of $\theta$ and $\psi$, respectively. We will occasionally emphasize this point using $\mathcal{P}_L$ and $P_L$ and, likewise, $Q_L$, $Q_L$.</footnote>

<footnote n="4">⁴ The careful reader will have realized that in this expression we suppress the *p* dependence in *g*, $\psi$'s and $\mathcal{P}$.</footnote>
<figure><img src="image_4.png" /><figcaption>**Figure 1.** (a) Examples of four percolation clusters of size $L^2 = 100^2$, obtained for $p = 0.2 < p_c$, $p = 0.6 > p_c$ in the top row and $p = 0.5$, i.e. just below $p_c$, in the bottom row. Occupied sites are marked by small dots while empty sites are left white. Each cluster of connected sites has been identified through the Hoshen–Kopelman algorithm. While individual clusters have been highlighted with different gray scales for the first three images, the bottom right image with $p = 0.5$ shows all occupied sites in black only, irrespective of cluster identity. This latter representation is used below for the ML approach. (b) Probabilities $P(p)$ and $Q(p)$ of having a spanning cluster (S, blue open squares) / or not (N, red open circles) close to the percolation threshold for dataset $T \cup V$, respectively. Similarly, $N_\tau$ (orange +) and $S_\tau$ (cyan ×) for probabilities $P(p)$, $Q(p)$ obtained for the test data set $\tau$. (c) Averaged correlation length $\langle \xi(p) \rangle$. In (b+c), the vertical lines indicate the estimates $p_c(100) \sim 0.585(5)$ (dashed) and $p_c$ (dotted).</figcaption></figure>

the extracted $P_{100}$, $Q_{100}$, $p_c(100)$ and $\langle \xi(p) \rangle$ estimates. The different gray scales displayed in figure 1(a) mark the different connected clusters. However, for the ML approach below, we shall only use the numerical values 0 and 1 corresponding to the state $\psi_i(p)^5$. This is visualized as the simple black and white version shown, e.g. for $p = 0.5$ in figure 1(a). From figures 1(b) and (c), we note that $P(p)$ and $\xi(p)$ behave qualitatively as expected [24], with $P_{100}(p) \gtrsim 0$ for $p < p_c$, $P_{100}(p) \lesssim 1$ for $p > p_c$ and $\xi(p)$ maximal near $p_c$. Clearly, $p_c(L = 100) \sim 0.585(5) < p_c$. This latter behavior is as expected since $s_L(p) \le s_\infty(p)$, i.e. a cluster that seemingly spans an $L \times L$ finite square might still not span on an infinite system.

We emphasize that in the construction, we took care to only construct states such that for each $p$, the number of occupied sites is exactly $N_{\text{occ}} = p \times L^2$ and hence $p$ can be used as exact label for the supervised learning approach. We note that $p = N_{\text{occ}}/L^2$ can therefore also be called the percolation density. For the ML results discussed below, it will also be important to note that the spacing between $p$ values reduces when $p$ reaches 0.5 with the next $p$ value given by 0.55 and then 0.555. Similarly, the $p$ spacing increases as 0.655, 0.66, 0.7. We will later see that this results in some deviations from perfect classification/regression. For reference, we now have 12 values $p = 0.1, \dots, 0.58 < p_c(100)$ and 18 values $p = 0.59, \dots, 0.9 > p_c(100)$. We also note that the training set contains 92.7% of states without a spanning cluster below $p_c$ and 94.8% are spanning above $p_c$. Last, we have also generated a similar test set with $L = 200$. We find that our results do not change significantly when using this much larger data set and hence we will refrain from showing any result for these larger states in the following.

## 2.3. Supervised ML strategy for phase prediction

As discussed above, DL neural networks using the power of CNNs are among the preferred approaches when trying to identify phases in condensed matter systems [36–38]. Here, we shall use a RESNET18 [1] network with 17 convolutional and 1 fully-connected layers, pretrained on the IMAGENET dataset [39] as usual [36, 40, 41]. As basis of our ML implementation we use the PYTORCH suite of ML codes [42]. We train the RESNET18 on the percolation of 310 000 states, using a 90%/10% split into training and validation data, $T$ and $V$, respectively; this corresponds to $N_{\text{train}} = 279 000$ and $N_{\text{val}} = 31 000$. We concentrate on two supervised ML tasks. First, we classify percolation images according to (i) $p$, (ii) $\xi$ as well as (iii) spanning versus non-spanning. In the second task, we aim to predict $p$ and $\xi$ values via ML regression. In both tasks, the overall network architecture remains identical, we just adapt the last layer as usual [43]. For the classification the output layers have a number of neurons corresponding to the number of classes trained. For instance in the case of classification by density, this corresponds to the $C = 31$ $p$-values given above; for classification according to average correlation length, we have $C = 31 \langle \xi(p) \rangle$ values and finally for the spanning/non-spanning classification it is $C = 2$. When training for a regression the output layer has only one neuron giving continuous numerical predictions. However, the loss functions are different. Let $w$ denote the set of parameters (weights) of the RESNET and let $(\psi_i, \chi_i)$ represent a given image sample with $\chi_i$ its classification/regression target, i.e. classes $p$ or $\xi$, and also $\chi'_i$ the predicted values, $p'$ or $\xi'$. For classification

<footnote n="5">⁵ We note that the full cluster information is already coded in the spatial distribution of these gray intensities, hence giving much information to the DNNs.</footnote>
<figure><img src="image_5.png" /><figcaption>**Figure 2.** Examples of percolation images from the three special test sets with (a) $\tau_{dl}$, a percolating straight line from top to bottom, (b) $\tau_{rw}$, a percolating random path from top to bottom and (c) $\tau_{fb}$, a 'firebreak'-like cross of empty sites preventing percolation. For the sake of visibility, in (a+b) the connected path is highlighted in red. In all three cases, $p = 0.5$.</figcaption></figure>

of categorical data, the class names are denoted by a class index $c = 1, \dots, C$ and encoded as $\chi_{ck} = 1$ if $\chi_c = k$, 0 otherwise. Then, for the (multi-class) classification problem, we choose the usual cross-entropy loss function, $l_c(\mathbf{w}) = -\frac{1}{n} \sum_{k=1}^n \sum_{c=1}^C \chi_{ck} \log \chi'_{ck}(\mathbf{w}) + (1 - \chi_{ck}) \log[1 - \chi'_{ck}(\mathbf{w})]$, where $n$ is the number of samples, i.e. either $N_{\text{train}}$ or $N_{\text{val}}$ [36]. We use the ADADELTA optimizer and find that its standard learning rate of $\ell = 1$$^6$ produces good convergence [44]. Another good metric for the classification task is the accuracy $a$, which is the ratio of correct predictions over $n$. The loss function for the regression problem is given by the mean-squared error $l_r(\mathbf{w}) = \frac{1}{n} \sum_{k=1}^n [\chi_k - \chi'_k(\mathbf{w})]^2$ while $\ell$ and the optimizer remain the same. When giving results for $l_c$ and $l_r$ below, we always present those after averaging over at least ten independent training and validation cycles, i.e. with a different initial split of the data into mini-batches. We use the notation $\langle l_c \rangle$ and $\langle l_r \rangle$ to indicate this averaging. In the case of classification, we also represent the quality of a prediction by confusion matrices [36]. These graphically represent the predicted class labels as a function of the true ones in matrix form, with an error-free prediction corresponding to a purely diagonal matrix. For comparison of the classification and regression calculations, we use in both cases a maximum number of $\epsilon_{\max} = 20$ epochs.

Our ML calculations train with a batch size of 256 for classification and for regression. All runs are performed on NVIDIA Quadro RTX6000 cards.

## 2.4. Generating test data sets

We generate a test data set, $\tau$, of 1000 states for each of the 31 $p$-values, such that in total we have $N_{\tau} = 31000$. This test set is used to make all the confusion matrices given below. By doing this, we ensure that the performance of the trained DL networks is always measured on unseen data [37].

In addition, we generate three special test data sets. These data sets have been constructed to allow testing for the existence of the spanning cluster. The first special data set, $\tau_{sl}$, is made for the 27 $p$-values 0.5, 0.55, 0.555, ..., 0.66, 0.7 close to $p_c$ and again consists of 1000 states $\psi_i(p)$ for each $p$. After generating each $\psi_i(p)$, we add a straight line of occupied sites from top to bottom, while keeping $p$ constant by removing other sites at random positions. Obviously, every $\psi_i(p)$ in $\tau_{sl}$ therefore contains at least one spanning cluster by construction. As a consistency check to the performance of the ML networks, we also add two more $\psi_i$ without any connecting path for $p = 0.1$ and 0.2.

In the next set, $\tau_{rw}$, we start with the same 27 $p$-values for a new set of 27000 $\psi_i(p)$, but instead of the straight line, we add a directed random walk from top to bottom. As before, we conserve the overall density $p$ of occupied sites. Hence, every sample in $\tau_{rw}$ is spanning. We again add two $\psi_i$ for $p = 0.1$ and 0.2 without the connected random path.

Finally, the third special data set, $\tau_{fb}$, again contains 27000 lattices for the same previously mentioned 27 $p$-values, but in each of the states we apply random firebreak paths, horizontally and vertically, of unoccupied sites. This set is clearly non-spanning. Following the same logic as for $\tau_{sl}$ and $\tau_{rw}$, we add two spanning test samples above $p_c$ without the firebreak, namely, for $p = 0.8$ and 0.9. In all three cases, despite the modification in the lattices we ensure that $N_{\text{occ}} = p \times L^2$ and hence the occupation density is $p$. Examples of the three sets can be seen in figure 2.

<footnote>$^6$ We also optimized other hyperparameters and tested other optimizers such as ADAM but found no significant performance improvement.</footnote>
<figure><img src="image_6.png" /><figcaption>**Figure 3.** (a) Average confusion matrix for classification according to *p*. The dataset used is the test data $\tau$ and the models used for predictions are those corresponding to a minimal $l_{c,val}$. True labels for *p* are indicated on the horizontal axis while the predicted labels are given on the vertical axis. The color scale represents the number of samples in each matrix entry. (b) Dependence of losses $l_{c,train}$ and $l_{c,val}$ averaged over ten independent training seeds, on the number of epochs $\epsilon$ for classification according to *p*. The circles (red solid) denote $l_{c,train}$ while the squares (blue open) show $l_{c,val}$. The green crosses indicate the minimal $l_{c,val}$ for each of the ten trainings.</figcaption></figure>

# 3. Results

## 3.1. Classification of states labeled with density *p*

We use the density $p$ values as labels for the ML task of image recognition with the DL implementation outlined in section 2.3. After ten trainings with all 310 000 images for 20 epochs, we find on average a validation loss of $\langle l_{c,val} \rangle = 0.052 \pm 0.009$ (corresponding to an accuracy of $\langle a_{c,val} \rangle = 99.323\% \pm 0.003$). This is comparable to the very good image classification results shown on kaggle [45]. Figure 3(a) gives the resulting averaged confusion matrix. The dependence of the training and validation losses, $\langle l_{c,train} \rangle$ and $\langle l_{c,val} \rangle$, respectively, on the number of epochs, $\epsilon$, is shown in figure 3(b). From the behavior of the loss functions, we can see that $\langle l_{c,val} \rangle \ge \langle l_{c,train} \rangle$ until $\epsilon = 15$ after which both losses remain similar. This suggests that $\epsilon_{max} = 20$ for our DL approach is indeed sufficient and avoids over-fitting. Similarly, the confusion matrix is mostly diagonal with the exception of very few samples around the change of resolution in density, at $p \sim 0.555$ and 0.655, as commented before in section 2.1.

## 3.2. Prediction of densities *p* via regression

For the regression problem, we train the ResNet18 only for the nine evenly spaced densities $p = 0.1, 0.2, \dots, 0.9$. After training and validation with $\mathcal{T}$ and $\mathcal{V}$, respectively, we examine the states in $\tau$ and predict their $p$ values. In figure 4, we present the results with (a) indicating the fidelity of the predictions for each $p$-value and (b) showing good convergence of the losses $l_{r,train}$ and $l_{r,val}$. Clearly, the regression works very well for the nine trained $p$-values $p = 0.1, \dots, 0.9$ as well as the untrained values $0.55, 0.555, \dots, 0.655, 0.656$ close to $p_c(100)$. After reaching $\epsilon = 20$, we find that $\min_{\epsilon}[\langle l_{r,train} \rangle] = 0.0003 \pm 0.0002$ and $\min_{\epsilon}[\langle l_{r,val} \rangle] = (6.2 \pm 1.2) \times 10^{-5}$.

Each predicted average $\langle p' \rangle$ value comes from the 1000 states in $\tau$, further averaged over ten training/validation cycles as explained in section 2.3. In figure 4 we also show, for $p = 0.56$ and 0.65, how these 1000 individual predictions are distributed in the $(p, p')$ plane. Clearly, there is no spread in the $p$-direction since $p = 0.56$ and 0.65 are fixed. However, there is some spread in the $p'$-direction, while the average $\langle p' \rangle$ values very closely follow the $p = p'$ lines. We note that only untrained data furthest away from trained data, i.e. the untrained $p = 0.65$ and 0.655, show some small deviations from the perfect $p = p'$.

Overall, we can conclude that our CNN performs well for classification and regression tasks while $\mathcal{T}, \mathcal{V}$, and $\tau$ present appropriately structured data sets for these ML tasks in terms of data size.

## 3.3. Classification with correlation length $\xi$

We now turn our attention to studying image recognition when using the correlation lengths $\xi$, instead of $p$, as labels for the $\psi_i(p)$ states. One way to do this is to compute $\langle \xi \rangle$ for each of the $p$-values used during the density training and use it as label. While for the classification by $p$ the label value was identical to the actual
<figure><img src="image_7.png" /><figcaption>**Figure 4.** (a) Average prediction $\langle p' \rangle$ obtained for regression according to $p$. The $\langle p' \rangle$ on the solid $p = p'$ line corresponds to the solid bottom and left axes, while the same $\langle p' \rangle$ data lying in the dashed $p = p'$ line is associated with the dashed top and right axes. The latter corresponds to a detailed representation of the region around $p_c(L)$. For both data representations, the blue open diamonds denote $p$-values that have been used during the training and the green open circles show $p$-values that were not trained. The vertical cloud of small orange dots shows the spread of $(p, p')$ values for 1000 individual samples at $p = 0.56$ with the average $(p, \langle p' \rangle)$ indicated by an orange circle. In the same fashion, the magenta dots give $(p, p')$ for $p = 0.65$ with the average denoted by the magenta circle. (b) Dependence of losses $l_{r,train}$ and $l_{r,val}$ averaged as in figure 3 on the number of epochs $\epsilon$ for regression according to $p$. The circles (red solid) denote $l_{r,train}$ while the squares (blue open) show $l_{r,val}$. The green crosses show the minimal $l_{r,val}$ for each of the ten trainings.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>**Figure 5.** (a) Average confusion matrix for classification according to $\langle \xi \rangle$. The dataset used is the test data $\tau$ and the models used for predictions are those corresponding to a minimal $l_{c,val}$. (b) Dependence of losses $l_{c,train}$ and $l_{c,val}$ on the number of epochs $\epsilon$ for classification according to $\langle \xi \rangle$. We follow the same convention as in figures 3 and 4.</figcaption></figure>

density $p$ of a given state, now each state is labeled by $\langle \xi(p) \rangle$. This means that the actual $\xi$ of the state might be different from the label assigned. Since $\langle \xi(p) \rangle$ can be uniquely identified by $p$, this strategy should be in fact equivalent to the previous situation and the CNN should give us similar classification results. The results of such a classification are shown in figure 5 where similarly to figure 3 we present in (a) the average confusion matrix for the 31 $\langle \xi(p) \rangle$ values (cf also figure 1) and in (b) the evolution of losses during the training. We find a validation loss of $\min_{\epsilon}[\langle l_{c,val} \rangle] = 0.38 \pm 0.07$ (corresponding to a maximal accuracy of $\max_{\epsilon}[\langle a_{c,val} \rangle] = 87.12\% \pm 0.05$) and a highly diagonal confusion matrix, with only a small deviation that can be linked to the change in resolution in our data set above $p = 0.5$.

One might wish to interpret the above classification with $\langle \xi(p) \rangle$ as a success of the ML approach. However, let us reemphasize that it is fundamentally equivalent to simply changing labels while keeping the direct connection of the labels with $p$ unaltered. We now wish to obtain a classification of states via their $\xi$'s
<figure><img src="image_9.png" /><figcaption>**Figure 6.** Probability distributions for correlation lengths $\xi$ when (a) $p < p_c$ (with 12 p-values) and $p > p_c$ (18 p-values) with unbalanced $\Xi$ and the balanced counterparts $\Xi_n$ and $\Xi_w$ denoted by yellow, magenta and green, respectively. In each case, the frequency $f$ is normalized relative to the total number of $\xi$'s in each set, i.e. for (a) 120 000 in $\Xi$ and $\Xi_n$ and 6 × 3560 = 21 360 in $\Xi_w$ while for (b) there are 180 000 in $\Xi$ and $\Xi_n$ and 5 × 3077 = 15 385 in $\Xi_w$.</figcaption></figure>

which is independent of the $p$'s. In figure 6 we show the distribution $\Xi$ of the $\xi$'s in $\mathcal{T} \cup \mathcal{V}$. Clearly, the number of small $\xi$ values is larger than the number of $\xi$ values close to the maximal value of $\max[\xi] = 15.771$ (cf also figure 1(c) for the averaged $\langle\xi\rangle(p)$ values.). Hence simply using each $\xi$ as label for the corresponding $\psi_i$ would result in a biased dataset. We therefore reorganize the $\mathcal{T} \cup \mathcal{V}$ data set. This can be done in two ways. For the first reorganization we create bins with a constant number of 10 000 samples in each bin. We call this dataset $\Xi_n$. This results in a varying bin width. The second way to reorganize the data set is to keep the bin width constant while restricting the number of samples in each bin. We shall denote this reorganization as $\Xi_w$. Since $\xi(p)$ is non-monotonic in $p$, we split the reorganization into the case (i) $p < p_c$ with and (ii) $p > p_c$. We emphasize that the reorganized data sets consist of the same states as in $\mathcal{T} \cup \mathcal{V}$ but now have different labels according to the bin labels for $\Xi_n$ and $\Xi_w$. Furthermore, there is now no longer any direct connection of the new labels to the original $p$ densities.

In figure 7, we plot the resulting confusion matrices and losses. We see that the classification for $\Xi_w$ and $\Xi_n$ only results in large diagonal entries in the confusion matrices for small correlation lengths labels $\xi$. Overall, the classification for $\Xi_w$ is somewhat better than for $\Xi_n$ when away from $p_c(L)$. We attribute this to the uneven spread of $\xi$ values for the $\Xi_n$. Still, with overall 52.4% and 59.6% of states misclassified for $\Xi_w$ and $\Xi_n$, respectively, it seems clear that classification for correlation lengths must be considered unsatisfactory.

### 3.4. Regression with correlation length $\xi$

For the regression task with $\xi$, we proceed analogously to section 3.2. Again, we train the CNN for the individual correlation length $\xi_i$ corresponding to each $\psi_i \in \mathcal{T}$ for the nine densities $p = 0.1, \dots, 0.9$. We then compute the predictions of $\xi_i$ for all 31 densities in $\tau$. The results are shown in figure 8. We find that the network architecture which previously predicted the density quite accurately is now struggling to correctly predict $\xi$. A structure seems to exist in the predictions. By looking closely we notice that the network makes use of the density for its predictions: although the individual true $\xi_i(p)$ values have a large spread for the untrained p-values, the regression reassigns them all a predicted $\langle\xi'\rangle$ value similar to the average of the $\xi'$ from the two neighboring p values.

We have detailed the case $p = 0.56$ in figure 8(a). We find that while the spread of $\xi$ values is rather large, reflecting the fluctuations inherent when calculating $\xi$ close to $p_c$, the spread of $\xi'$ is much less, and $\langle\xi(0.56)'\rangle = 3.48$ is close to the average of the trained $\langle\xi'(0.5)\rangle$ and $\langle\xi'(0.6)\rangle$, namely 3.46. Similarly, the prediction at $p = 0.65$ is 2.14 and hence close to the average of $\langle\xi'(0.6)\rangle$ and $\langle\xi'(0.7)\rangle$, namely 1.91. We note that in obtaining these results, we have taken care to avoid unintentional feature leakage [46, 47] by removing information about $p$ in the $\tau$ data set.

### 3.5. Classification with the spanning or non-spanning properties

As discussed earlier, the hallmark of the percolation transition is the existence of a spanning cluster which determines whether the system is percolating or not [24]. In the previous section, our DL approach has classified according to $p$ or $\xi$ values without testing whether spanning clusters actually exist. We now want to
<figure><img src="image_10.png" /><figcaption>**Figure 7.** Confusion matrices of the τ set and losses $l_{c,train}$ and $l_{c,val}$ for the classification results when using the correlation-length-relabeled $\Xi_n$ and $\Xi_w$ data sets. The rows (a), (c) show the case $p < p_c$ while (b), (d) give the outcome for $p > p_c$. (a), (b) corresponds to 10000 states for each class with 12 classes for $p < p_c$ and 18 at $p > p_c$ for $\Xi_n$. For $\Xi_w$ in (c), (d), there are 3560 states for the 6 classes when $p < p_c$ and 3077 states for 5 classes when $p > p_c$.</figcaption></figure>

check this and label all states according to whether they are spanning or non-spanning. From figure 1, it is immediately clear that for finite-sized systems considered here, there are a non-negligible number of states which appear already spanning even when $p < p_c$ and, vice versa, are still non-spanning when $p > p_c$. Furthermore, we note that for such $L$, the difference between $p_c$ and $p_c(L)$ is large enough to be important and we hence use $p_c(L)$ as the appropriate value to distinguish the two phases.
<figure><img src="image_11.png" /><figcaption>**Figure 8.** (a) Average predictions for regression according to $\xi$. The dataset used is the test data $\tau$ and the models used for predictions are those corresponding to a minimal $l_{r,val}$. The cloud of small magenta dots shows the spread of $(\xi_i, \xi_i')$ values for 1000 individual samples at $p = 0.65$ with averages $(\langle\xi\rangle, \langle\xi'\rangle)$ indicated by a magenta circle. In the same fashion, the orange dots give $(\xi_i, \xi_i')$ for $p = 0.56$ with averages denoted by the orange circle. (b) Dependence of losses $l_{r,train}$ and $l_{r,val}$ on the number of epochs $\epsilon$ for regression according to $\xi$. We follow the same color/symbol convention as in figure 4.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>**Figure 9.** (a) Average confusion matrix for classification according to spanning/non-spanning. The dataset used is the test data $\tau$ and the models used for predictions are those corresponding with a minimal $l_{r,val}$. The true labels for N and S, are indicated on the horizontal axis while the predicted labels are given on the vertical axis. (b) Dependence of losses $l_{c,train}$ and $l_{c,val}$ on the number of epochs $\epsilon$ for classification according to spanning/non-spanning. Again, we follow the same convention as for figures 3, 5 and 7.</figcaption></figure>

Figure 9 shows the average results after $\epsilon = 20$ with an validation loss of $\min_{\epsilon}[\langle l_{c,val} \rangle] = 0.165 \pm 0.001$ (corresponding to a maximal validation accuracy $\max_{\epsilon}[\langle a_{c,val} \rangle] = 92.702\% \pm 0.001$).

At first glance, the figure seems to indicate a great success: from the 31 000 states present in $\tau$, 11 510.6 have been correctly classified as non-spanning (i.e. $N \to N'$), and 17 206.9 as spanning ($S \to S'$) while only 1223.1 are wrongly labeled as non-spanning ($S \to N'$) and 1059.4 as spanning ($N \to S'$)$^7$. Overall, we would conclude that 92.6% of all test states are correctly classified while 7.4% are wrong.

However, from the full percolation analysis for $\tau$, we can compute that there are 11 127 states (92.7%) without a spanning cluster below $p_c(L)$ while 873 states (7.3%) already contain a spanning cluster. Similarly, for $p > p_c(L)$, 94.9% of states, equivalent to 17 075 states, are spanning and 5.1% are not, corresponding to 925 states. At $p_c(L) = 0.585$, we furthermore have 482 spanning and 518 non-spanning states. Hence in total, we expect 2280 wrongly classified states. Since the last number is decisively close to the actual number of

<footnote>$^7$ We note that these numbers are not integers since they are computed as averages over the ten independent training runs as mentioned in section 2.3.</footnote>
<figure><img src="image_13.png" /><figcaption>**Figure 10.** (a) The blue curve (red curve) shows the probability to have a spanning (non-spanning) sample in the training dataset. The cyan (orange) curve gives us the prediction of probability to have a spanning (non-spanning) sample, according to the trained network. (b) Dependence of the Pearson correlation coefficient *r* on the density *p* for classification according to spanning/non-spanning. The confidence interval is indicated in gray. In both (a) and (b), the lines connecting the symbols are only a guide to the eye.</figcaption></figure>

2282.5 of misclassified states, this suggests that it is precisely the spanning states below $p_c(L)$ and the non-spanning ones above $p_c(L)$ which the DL network is unable to recognize. Let us rephrase for clarity: it seems that the DL CNN, when trained in whether a cluster is spanning or non-spanning, completely disregards this information in its classification outputs.

### 3.6. Density-resolved study of spanning/non-spanning close to $p_c(L)$

In order to understand the behavior observed in the last section, we now reexamine the result of figure 9 by analyzing the ML-predicted probabilities, $P_{\text{ML}}(p)$. In figure 10, we show both $P_{\text{ML}}(p)$ as well as $P(p)$; the latter having been obtained by the Hoshen-Kopelman algorithm, cf figure 1(a). While the $P(p)$ and $P_{\text{ML}}(p)$ curves—and of course also the corresponding $Q(p)$ and $Q_{\text{ML}}(p)$— appear qualitatively similar, they are nevertheless not identical and the slopes of $P_{\text{ML}}(p)$, $Q_{\text{ML}}(p)$ are different. We emphasize that the slopes are important for determining the universality class of a second-order phase transition via finite-size scaling [48]. Since we know for each image whether it percolates or not, we can also check how well the ML predictions worked by considering the covariance. Let $\zeta(\psi_i(p)) = 0$ when there is no percolating cluster in the state $\psi_i(p)$ while $\zeta(\psi_i(p)) = 1$ if there is. Similarly, we define $\zeta_{\text{ML}}(\psi_i(p))$ for the prediction by the DL network. Then $\text{cov}(\zeta, \zeta_{\text{ML}})(p)$ measures the covariance of states being found to span by percolation and by ML for given $p$. In figure 10(b) we show the normalized result, i.e. the Pearson coefficient $r_{\zeta, \zeta_{\text{ML}}}(p) = \text{cov}(\zeta, \zeta_{\text{ML}})(p)/[\sigma_{\zeta}(p)\sigma_{\zeta_{\text{ML}}}(p)]$, where $\sigma_{\zeta}$ and $\sigma_{\zeta_{\text{ML}}}$ are the standard deviations of the percolation results and the ML predictions. We see that in the transition region, $r_{\zeta, \zeta_{\text{ML}}} \lesssim 0.12$ which is very far from the maximally possible value 1. This suggest that while the ML predictions are not simply random, they are also not very much better than random.

Let us now study the classification into spanning/non-spanning states in detail for each $p$. Figure 11 and table 1 show a comparison of the classification for the ten $p$ values 0.56 to 0.605. We see, e.g. that for $p = 0.56, 0.565, 0.57, 0.575 < p_c(L) \sim 0.585, 485 (=48+87+126.5+223.5)$ of 492 (=48+87+127+230) samples, which are already spanning, have been misclassified as non-spanning. Similarly, for $p = 0.59, 0.595, 0.6, 0.605 > p_c(L)$, 745.9 of in total 864 still non-spanning samples are classified as spanning. These results are similar whether one considers a typical sample or the averaged result. Hence, contrary to the supposed success of figure 9, we now find that the seemingly few misclassified states of figure 9 are indeed precisely those which represent the correct physics. Saying it differently, the ML process seems to have led to a DL network which largely disregards the characteristic of spanning clusters and just uses the overall density of occupied vs. non-occupied sites to ascertain the phases. Of course, this is the wrong physics when considering percolation.

### 3.7. Testing the accuracy of the DL network

The difficulties that the trained DL network has with recognizing whether a state contains a percolating cluster or not can be made more explicit. In section 2.4, we had generated three test sets for this purpose. Namely, percolating states even for $p < p_c(L)$ by adding (i) a straight line, $\tau_{\text{sl}}$, and (ii) a random walk, $\tau_{\text{rw}}$, of connecting sites as well as for $p > p_c(L)$ (iii) the firebreak states, $\tau_{\text{fb}}$, of percolation-prohibiting random unoccupied sites. We now use these sets and feed them independently as test sets to the DL network. Figure 12 shows the three confusion matrices obtained when classifying for spanning vs. non-spanning. In figures 12(a) and (b), we see that the network completely misclassifies the spanning datasets $\tau_{\text{sl}}$ and $\tau_{\text{rw}}$. The
<figure><img src="image_14.png" /><figcaption>**Figure 11.** Confusion matrices showing the predictions of the trained network figure 9 in a region $p = [0.56, 0.605]$ comprising $p_c$, with (a) for predictions made before the percolation threshold, (b) in the threshold region and (c) after the percolation threshold. Each confusion matrix is an average of the predictions made by the ten trained models shown in figure 9(b).</figcaption></figure>

<table><caption>**Table 1.** Predictions of the trained network on the test data set $\tau$ with $n = 1000$ for $p \in [0.56, 0.605]$. N and S denote, respectively, the number of non-spanning and the number of spanning samples in $\tau$. The four following columns $S \to S'$, $S \to N'$, $N \to S'$, and $N \to N'$ give the averaged results of 10 independent prediction runs.</caption><thead><tr><th rowspan="2">p</th><th colspan="2">N</th><th colspan="2">S</th><th colspan="2">S → S'</th><th colspan="2">S → N'</th><th colspan="2">N → S'</th><th colspan="2">N → N'</th></tr><tr><th>#</th><th>%</th><th>#</th><th>%</th><th>⟨#⟩</th><th>⟨%⟩</th><th>⟨#⟩</th><th>⟨%⟩</th><th>⟨#⟩</th><th>⟨%⟩</th><th>⟨#⟩</th><th>⟨%⟩</th></tr></thead><tbody><tr><td>0.56</td><td>952</td><td>95.2</td><td>48</td><td>4.8</td><td>0.0</td><td>0.0</td><td>48.0</td><td>4.8</td><td>0.0</td><td>0.0</td><td>952.0</td><td>95.2</td></tr><tr><td>0.565</td><td>913</td><td>91.3</td><td>87</td><td>8.7</td><td>0.0</td><td>0.0</td><td>87.0</td><td>8.7</td><td>0.4</td><td>0.0</td><td>912.6</td><td>91.3</td></tr><tr><td>0.57</td><td>873</td><td>87.3</td><td>127</td><td>12.7</td><td>0.5</td><td>0.1</td><td>126.5</td><td>12.7</td><td>1.8</td><td>0.2</td><td>871.2</td><td>87.1</td></tr><tr><td>0.575</td><td>770</td><td>77.0</td><td>230</td><td>23.0</td><td>6.5</td><td>0.7</td><td>223.5</td><td>22.4</td><td>13.1</td><td>1.3</td><td>756.9</td><td>75.7</td></tr><tr><td>0.58</td><td>646</td><td>64.6</td><td>354</td><td>35.4</td><td>51.0</td><td>5.1</td><td>303.0</td><td>30.3</td><td>55.7</td><td>5.6</td><td>590.3</td><td>59.0</td></tr><tr><td>0.585</td><td>518</td><td>51.8</td><td>482</td><td>48.2</td><td>218.8</td><td>21.9</td><td>263.2</td><td>26.3</td><td>181.5</td><td>18.2</td><td>336.5</td><td>33.6</td></tr><tr><td>0.59</td><td>380</td><td>38.0</td><td>620</td><td>62.0</td><td>512.8</td><td>51.3</td><td>107.2</td><td>10.7</td><td>279.0</td><td>27.9</td><td>101.0</td><td>10.1</td></tr><tr><td>0.595</td><td>232</td><td>23.2</td><td>768</td><td>76.8</td><td>737.1</td><td>73.7</td><td>30.9</td><td>3.1</td><td>217.0</td><td>21.7</td><td>15.0</td><td>1.5</td></tr><tr><td>0.60</td><td>169</td><td>16.9</td><td>831</td><td>83.1</td><td>824.9</td><td>82.5</td><td>6.1</td><td>0.6</td><td>167.0</td><td>16.7</td><td>2.0</td><td>0.2</td></tr><tr><td>0.605</td><td>83</td><td>8.3</td><td>917</td><td>91.7</td><td>916.3</td><td>91.6</td><td>0.7</td><td>0.1</td><td>82.9</td><td>8.3</td><td>0.1</td><td>0.0</td></tr></tbody></table>

two correctly identified non-spanning images are just the two such states added to each of the data sets to show that the network is still performing. Similarly, in figure 12(c), we see that this time the network cannot correctly identify the non-spanning samples in $\tau_{fb}$. Again, the two samples correctly identified are the ones without the firebreak.
<figure><img src="image_15.png" /></figure>

4. Conclusions

Let us briefly summarize what we have achieved thus far. We showed that when looking at *p*, classification and regression techniques for percolation states allow us to obtain good recognition with near-perfect $\langle a_{c,val} \rangle = 99.323\% \pm 0.003$) for classification and near-zero $\langle l_{r,val} \rangle = 0.000\,062 \pm 0.000\,012$ average mean-square loss for regression. Confusion matrices are heavily diagonal for classification while prediction curves for regression are similarly convincing. These results are in good agreement with previous similar such studies [25, 26, 29], which should not come as a surprise: the information about *p* is of course directly enclosed in each state. We emphasize that our approach is already somewhat more challenging than these previous works since instead of just asking the DL network to identify the two phases $p < p_c$ and $p > p_c$, we also successfully identify all 31 distinct densities *p*. Using $\langle \xi(p) \rangle$ instead of *p* to identify the 31 densities also works quite well, but is again expected since $\langle \xi(p) \rangle$ merely acts as a new set of supervised labels.

Problems emerge when we use the computed correlation length $\xi$ for each state and try classification and regression with these correlation lengths. Having thus explicitly removed any connection with the *p* values, we nevertheless find that the resulting confusion matrices, fidelity curves and losses are all of much less quality than before. Instead, it seems that the density *p* information is still the overriding measure used by the DL network to arrive at its outputs (cf figures 7 and 8)<fnref n="8" />. Rather, as we show in sections 3.5–3.7, the DL network completely ignores whether a cluster is spanning or non-spanning, essentially missing the underlying physics of the percolation problem—it seems to still use *p* as its main ordering measure.

We believe that the root cause of the failure to identify the spanning clusters, or their absence, lies in the fundamentally local nature of the CNN: the filter/kernels employed in the RESNETs span a few *local* sites only<fnref n="9" />. Hence it is not entirely surprising that such a CNN cannot correctly identify the essentially *global* nature of spanning clusters. But it is of course exactly this global percolation that leads to the phase transition.

The reader might wonder why previous CNN studies of phases in other models, such as, e.g. the Ising-type models [9], the three-dimensional Anderson model and its topological variants [12], have failed to find similar such issues. We think that this is because in the Ising case, the majority rule for spin alignment is not concerned with any globally spanning domain [49], while in the Anderson-type models, it is the (typical) local density of states which can serve as order parameter [50]. In short, in these models a local property is indeed sufficient to distinguish their phases.

Of course ML aficionados might now want to suggest that extensions of local kernels are possible in CNNs. Indeed, one might, e.g. want to use CNNs in which large dilution parameters are employed to effectively make filters/kernels of manageable size while still spanning across a sizeable portion of the $L \times L$ size of each percolation state [51]. But while this might solve the issue for fixed *L* in the percolation case it does imply knowing that a global property is important to start with. This would rather diminish the relevance of DL as a tool for unbiased discovery in physics. We note that, after submission of our original manuscript, such a strategy seems to have been pursued in [52].

<footnote n="8">⁸ We emphasize that we have spent considerable effort at making sure that this result is not due to erroneous information leakage [37].</footnote>

<footnote n="9">⁹ In addition to the RESNET18 used in the main text, we have also checked that RESNET34 yields a similar outcome.</footnote>
## Data availability statement

The data that support the findings of this study are openly available at the following URL/DOI: https://wrap.warwick.ac.uk/175178/.

## Acknowledgments

We thank B Çivitcioğlu, M Hilke, and T Ohtsuki for discussions. DB is grateful for co-tutelle funding via the EUtopia Alliance. RAR gratefully acknowledges funding by the CY Initiative of Excellence (grant "Investissements d'Avenir" ANR-16-IDEX-0008) and hospitality during various stays at CY Advanced Studies. We gratefully acknowledge the University of Warwick Research Technology Platform (RTP Scientific Computing) and the Sulis Tier 2 HPC platform hosted by the RTP. Sulis is funded by EPSRC Grant EP/T022108/1 and the HPC Midlands+ consortium. UK research data statement: Data accompanying this publication are available in [53].

## ORCID iDs

Djénabou Bayo https://orcid.org/0009-0000-4030-922X

Andreas Honecker https://orcid.org/0000-0001-6383-3200

Rudolf A Römer https://orcid.org/0000-0002-8719-3313

## References

[1] He K, Zhang X, Ren S and Sun J 2016 Proc. IEEE Comput. Soc. Conf. on Computer Vision and Pattern Recognition (December 2016) pp 770–8

[2] Tabak M A et al 2019 Methods Ecol. Evol. 10 585–90

[3] Zhang R, Wang Q and Lu Y 2017 Combination of ResNet and center loss based metric learning for handwritten Chinese character recognition 2017 14th IAPR Int. Conf. on Document Analysis and Recognition (ICDAR) (IEEE) pp 25–29

[4] Zhao Y and Khushi M 2020 IEEE Int. Conf. on Data Mining Workshops (ICDMW 2020) pp 385–91

[5] Nomura Y, Darmawan A S, Yamaji Y and Imada M 2017 Phys. Rev. B 96 205152

[6] Rao W J 2018 J. Phys.: Condens. Matter 30 395902

[7] Zhang Y and Kim E A 2017 Phys. Rev. Lett. 118 216401

[8] Stoudenmire E and Schwab D J 2016 Supervised learning with tensor networks Advances in Neural Information Processing Systems vol 29, ed D Lee, M Sugiyama, U Luxburg, I Guyon and R Garnett (Curran Associates, Inc.)

[9] Carrasquilla J and Melko R G 2017 Nat. Phys. 13 431–4

[10] van Nieuwenburg E P L, Liu Y H and Huber S D 2016 Nat. Phys. 13 435–9

[11] Venderley J, Khemani V and Kim E A 2018 Phys. Rev. Lett. 120 257204

[12] Ohtsuki T and Mano T 2020 J. Phys. Soc. Japan 89 022001

[13] Ching K, Carrasquilla J, Melko R G and Khatami E 2017 Phys. Rev. X 7 031038

[14] Käming N, Dawid A, Kottmann K, Lewenstein M, Sengstock K, Dauphin A and Weitenberg C 2021 Mach. Learn.: Sci. Technol. 2 035037

[15] Saito H 2017 J. Phys. Soc. Japan 86 093001

[16] Saito H 2018 J. Phys. Soc. Japan 87 074002

[17] Cai Z and Liu J 2018 Phys. Rev. B 97 035116

[18] Carleo G and Troyer M 2017 Science 355 602–6

[19] Vicentini F et al 2022 SciPost Phys. Codebases 7

[20] Huang L and Wang L 2017 Phys. Rev. B 95 035105

[21] Liu J, Qi Y, Meng Z Y and Fu L 2017 Phys. Rev. B 95 041101

[22] Luo C, Keshtegar B, Zhu S P, Taylan O and Niu X P 2022 Comput. Methods Appl. Mech. Eng. 388 114218

[23] Broadbent S R and Hammersley J M 1957 Math. Proc. Camb. Phil. Soc. 53 629–41

[24] Stauffer D and Aharony A 1991 Introduction to Percolation Theory 2nd edn (Taylor & Francis Group)

[25] Zhang W, Liu J and Wei T C 2019 Phys. Rev. E 99 032142

[26] Shen J, Li W, Deng S and Zhang T 2021 Phys. Rev. E 103 052140

[27] Patwardhan S, Majumder U, Sarma A D, Pal M, Dwivedi D and Panigrahi P K 2022 Machine learning as an accurate predictor for percolation threshold of diverse networks (arXiv:2212.14694)

[28] Yu W and Lyu P 2020 Physica A 559 125065

[29] Cheng S, He F, Zhang H, Zhu K D and Shi Y 2021 Machine learning percolation model (arXiv:2101.08928)

[30] Elliott R J, Heap B R, Morgan D J and Rushbrooke G S 1960 Phys. Rev. Lett. 5 366–7

[31] Flory P J 1941 J. Am. Chem. Soc. 63 3083–90

[32] Derrida B and Stauffer D 1985 J. Phys. France 46 1623–30

[33] Grimmett G 1989 Percolation (Springer)

[34] Jacobsen J L 2014 J. Phys. A: Math. Theor. 47 135001

[35] Hoshen J and Kopelman R 1976 Phys. Rev. B 14 3438–45

[36] Mehta P, Bukov M, Wang C H, Day A G R, Richardson C, Fisher C K and Schwab D J 2019 Phys. Rep. 810 1–124

[37] Dawid A et al 2022 Modern applications of machine learning in quantum sciences (arXiv:2204.04198)

[38] Bedolla E, Padierna L C and Castañeda-Priego R 2020 J. Phys.: Condens. Matter 33 053001

[39] Deng J, Dong W, Socher R, Li L J, Li K and Fei-Fei L 2009 2009 IEEE Conf. on Computer Vision and Pattern Recognition pp 248–55

[40] Chollet F 2018 Deep Learning With Python (IEEE)
[41] Han X *et al* 2021 *AI Open* 2 225–50

[42] Paszke A *et al* 2019 *Advances in Neural Information Processing Systems* vol 32

[43] SciKit-Learn 1.2.2: 1.17. Neural network models (supervised) 2023 (available at: https://scikit-learn.org/stable/modules/neural_networks_supervised.html)

[44] Zeiler M D 2012 ADADELTA: an adaptive learning rate method (arXiv:1212.5701)

[45] Kaggle competition 'Dogs vs. Cats': create an algorithm to distinguish dogs from cats 2012 (available at: www.kaggle.com/competitions/dogs-vs-cats/leaderboard)

[46] Kaufman S, Rosset S, Perlich C and Stitelman O 2012 *ACM Trans. Knowl. Discovery Data* 6 1–21

[47] Kapoor S and Narayanan A 2023 *Patterns* 4 100804

[48] Kirkpatrick T R and Belitz D 1992 *J. Phys.: Condens. Matter* 4 L37–L42

[49] McCoy B M and Wu T T 1973 *The Two-Dimensional Ising Model* (Harvard University Press)

[50] Dobrosavljević V, Pastor A A and Nikolić B K 2003 *Europhys. Lett.* 62 76–82

[51] Al-Shabi M, Lee H K and Tan M 2019 *IEEE Access* 7 178827–38

[52] Oh S M, Choi K and Kahng B 2023 *J. Stat. Mech.* 2023 083210

[53] Bayo D, Honecker A and Römer R A 2023 Data for ‘The percolating cluster is invisible to image recognition with deep learning’ (available at: https://wrap.warwick.ac.uk/175178/)