# Assessing the quality of whole slide images in cytology from nuclei features

Paul Barthe, Romain Brixtel, Yann Caillot, Benoît Lemoine, Arnaud Renouf, Vianney Thurotte, Ouarda Beniken, Sébastien Bougleux, Olivier Lézoray

► To cite this version:

Paul Barthe, Romain Brixtel, Yann Caillot, Benoît Lemoine, Arnaud Renouf, et al. Assessing the quality of whole slide images in cytology from nuclei features. Journal of Pathology Informatics, 2025, 17, pp.100420. 10.1016/j.jpi.2025.100420. hal-04964441

HAL Id: hal-04964441

https://hal.science/hal-04964441v1

Submitted on 24 Feb 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# Assessing the quality of whole slide images in cytology from nuclei features

<figure><img src="image_3.png" /></figure>

Check for
updates

Paul Barthe<sup>a,b,*</sup>, Romain Brixtel<sup>a,*</sup>, Yann Caillot<sup>a</sup>, Benoît Lemoine<sup>a</sup>, Arnaud Renouf<sup>a</sup>, Vianney Thurotte<sup>a</sup>, Ouarda Beniken<sup>a</sup>, Sébastien Bougleux<sup>b,*</sup>, Olivier Lézoray<sup>b</sup>

## ARTICLE INFO

**Keywords:**

Automated quality control

Digital pathology

Cytology

Whole slide image analysis

Out-of-distribution detection

## ABSTRACT

**Background and objective:** Implementation of machine learning and artificial intelligence algorithms into digital pathology laboratories faces several challenges, notably the variation in whole slide image preparation protocols. The diversity of preparation pipelines forces algorithms to be protocol-dependant. Moreover, the error susceptibility of each stage in the preparation process implies a need of quality control tools. To address these challenges, this article introduces a straightforward, interpretable, and computationally efficient quality control module to ensure optimal algorithmic performance.

**Methods:** The proposed quality control module ensures algorithmic performance by representing an algorithm by a reference whole slide image preparation protocol validated on it. Then, inspired by data description methods, a preparation protocol is represented by nuclei feature distributions, obtained for several whole slide images it has produced. The quality of a preparation protocol is evaluated according to several reference preparation protocols, by comparing their feature distributions with a weighted distance.

**Results:** Through empirical analysis conducted on seven distinct preparation protocols, we demonstrated that the proposed method build a quality module that clearly discriminates each preparation. Additionally, we showed that this module performs well on more larger and realistic corpus from laboratories routine, detecting quality deviations.

**Conclusion:** Even if the proposed method necessitates minimal data and few computational resources, we showed that it is interpretable and relevant on realistic corpus from several laboratories' routine. We strongly believe in the necessity of quality control from the algorithmic perspective and hope this kind of approach will be extended to improve quality and reliability of digital pathology whole slide images.

## Introduction

The Digital Pathology Association defines digital pathology (DP) as encompassing tools and systems for digitizing pathology slides and associated metadata, including their storage, review, and analysis, as well as the enabling infrastructure.¹ WSIs are the numerical representation of glass slides and are often very large files, sometimes exceeding a gigabyte, which poses challenges for storage and transfer. This is why, in recent decades, advances in computational technology and storage have provided opportunities for DP to grow quickly. Computer-aided diagnosis (CAD) systems play a crucial role in the review and analysis of whole slide images (WSIs), aiming to enhance diagnostic accuracy and efficiency of pathology experts by using the computer output as a guide. Such systems and, in turn, DP laboratories, have recently benefited from advances in artificial intelligence (AI) and machine learning (ML). For that reason, DP is now sometimes used as a more “generic term that includes AI-based approaches for detection, segmentation, diagnosis, and analysis of WSIs.”³⁴ Despite

these advancements, the implementation of CAD systems within a WSI preparation pipeline (WSI-PP) is hindered by the diversity of preparation protocols used. Different protocols result in WSIs with various characteristics, necessitating the customization of CAD algorithms to match these unique features. Before using an algorithm, laboratories must then check that their WSI-PPs meet the algorithm's requirements, in order to achieve optimal algorithmic performance. This article specifically addresses such needs of quality control (QC) for cytological laboratories.

### Digital cytology laboratory workflow

Digital cytology is a branch of DP focusing on the analysis of cells in biological samples. To better understand why cytological WSIs are prone to quality errors, we briefly describe the four main stages of a digital cytology laboratory workflow: sample collection, sample preparation, sample coloration, and slide digitization. As described by Mach et al.,³¹ sample collection includes phlebotomy, fluid aspiration, or saline washing of a mucosal

<footnote><sup>a</sup> *Datexim, Caen 14000, France*</footnote>

<footnote><sup>b</sup> *Normandie Univ, UNICAEN, ENSICAEN, CNRS, GREYC, Caen 14000, France*</footnote>

<footnote>* Corresponding authors.</footnote>

<footnote>E-mail addresses: paul.barthe@datexim.ai (P. Barthe), romain.brixtel@datexim.ai (R. Brixtel), sebastien.bougleux@unicaen.fr (S. Bougleux).</footnote>
surface with catheter assistance. Once the sample is collected, an aliquot of
this biofluid is centrifuged or filtered, usually with an automated instru-
ment, to collect cell sediments and evenly distribute them on a glass slide.
Subsequently, automated systems stained the slide with specific dyes and
either alcohol-fixed or air-dried to remove water content before the glass
cover strip is applied. Finally, scanners are used to digitized the glass
slide into a WSIs.

WSI analysis

As cytology involves cell analysis, typically via the nucleus, we focus here on the WSI analysis workflow from a nuclei point of view. This classical workflow³¹ can be divided into four stages, as depicted in Fig. 1. First, nuclei detection is performed on the WSI to identify areas in which the nuclei are located. Following this, fine-grained information about detected object boundaries is provided via segmentation. Then, segmented objects can be filtered by any heuristic and features are calculated and assigned to each resulting object. The analysis of these segmented objects is then conducted by a processing module, which may utilize various approaches (e.g., expert systems or machine learning) and pursue different objectives (e.g., transformation, feature analysis, or classification). It should be noted that henceforth in this article, the term nuclei will refer to segmented objects. However, it is known that a segmented object equates to a nucleus only if the object detection and segmentation are flawless.

Quality control

As mentioned above, the analysis of WSIs can only achieve optimal al-
gorithmic performance if the WSI-PP meets the algorithm's requirements.
However, each stage of the WSI-PP can affect the quality of the WSI. For
sample collection, depending on the technique and the surgeon, the sam-
ple can vary in appearance, and artifacts may be introduced. For sample
preparation, each automated system manufacturer has its own settings
and operational methods, which produce staining with different charac-
teristics. For slide digitization, numerous WSI scanners are available on
the market, each with its own hardware and software components, lead-
ing to significant variations in WSIs characteristics. However, as high-
lighted by Janowczyk et al.,¹⁸ manual review of glass and digital slides
is laborious, qualitative, and subject to intra- and inter-reader variability,
underscoring the need for an automated and reproducible method to as-
sess WSI quality. In response to these challenges, we propose a straightfor-
ward, interpretable and computationally efficient QC module, to assess
WSI-PP quality with respect to (w.r.t.) reference WSI-PPs. This module
is capable of raising quality alerts during its creation, prompting a recon-
sideration of reference WSI-PPs. Once created, the module measures the
similarity between reference preparation pipelines, representing algo-
rithms, and the tested preparation pipeline. If the used measure is not sat-
isfactory, an alert is raised.

In next section, we explore and show limitations of existing research,
highlighting the need for further advancements. To address this gap, third
section details the method pursued to build our QC module and introduce
the material used to: empirically determine optimal parameter values for
the QC module and confront this module to real-world production environ-
ments. Finally, we present experiments and discuss results of our QC mod-
ule on real-world production environments.

Related works

WSI quality is highly dependent on the WSI-PP. Each stage can intro-
duce severe artifacts (unexpected objects), prejudicial for human expert in-
terpretation and representing potential algorithm failure cases. Identifying
artifacts in a pre-processing step is then required. We review QC pipelines
created to handle artifacts from preparation and digitization. Then, we in-
troduce QC methods that have been designed for WSI-PP and highlight
the fact that QC from an algorithm admissibility perspective is poorly rep-
resented in the DP literature. We then discuss about the importance of
such QC and present works dealing with the out-of-distribution (OOD)
problem, which are more akin to our proposed method.

Quality control modules

The importance of implementing QC is illustrated in Song et al.,⁴⁶ where they trained a deep network on histopathological WSIs digitized with a specific scanner. Then, they digitized a WSI test set with three scanner models, including the scanner used during the training stage, and inferred the network on this set. They observed a significant difference in performances between the scanner used to create the training set and the two other scanners. Moreover, Ogura et al.³⁵ showed that even the same scanner instance on the same WSI can produce different results at different times. It is in response to these concerns that QC modules have been built.

A QC module, based on sharpness features extracted from cells, is pro-
posed by Lahrmanb et al.²³ A WSI sharpness score is used jointly with a
user-defined threshold in order to consider a WSI as analyzable. In Ameisen
et al.,² a WSI is considered for further use according to scores evaluating tile
blurriness, contrast, brightness, and color at several magnification levels.
Campanella et al.⁵ used sharpness measures from patches jointly with a ran-
dom forest classifier to provide a blur slide-level score. Another slide-level
score, is introduced by Zhang et al.,⁵² with the use of a CNN. Shakhawat
et al.⁴¹ present a quality evaluation method which distinguishes two origins
of quality degradation, preparation or digitization. They used a support vec-
tor machine (SVM) classifier to detect air bubbles and tissue folds artifacts
in histological WSIs in order to inform on slide preparation quality. For dig-
itization, only WSI regions without artifacts are analyzed. Then, based on
Hashimoto et al.,¹⁶ sharpness and noise features are computed on artifact-
free zones and fed to a linear regression model to evaluate slide quality. Ad-
ditionally to the development of a multi-class deep learning model for se-
mantic segmentation artifacts, Smit et al.⁴³ proposed a QC module that
informs on the actions that must be done to fix artifacts' issues. In Haghighat
et al.,¹³,¹⁴ a WSI is accepted or rejected according to a slide-level quality
score based on predicted patch-level quality measures from a multi-label
deep neural network.

We present in the following a framework, designed for QC modules, and
place these studies into it in order to highlight the fact that a specific QC is
not represented.

Integrating quality control in digital pathology

Given the importance of QC for preparation pipelines, several studies
have examined the limitations of AI applied to DP.³,⁸,³⁹ They discussed
how advances in AI should be considered and the challenges that need to
be overcome before the adoption of AI, in particular for ML models.

<figure><img src="image_4.png" /><figcaption>Fig. 1. Overview of a typical cytological WSI analysis workflow. The analysis module icon represents an analysis with a classification objective.</figcaption></figure>
QC modules have also been designed, like in Smith et al.,⁴⁴ where they discussed about issues in WSIs' analysis before dawning up a QC module framework for deep learning project. More generically, Brixtel et al.⁴ suggested integrating QC into every stage of the DP diagnosis pipeline, and proposed a QC module framework. Fig. 2 is taken from their article and slightly modified it to suit our needs. In this framework, two QC checks are performed on two different nodes, one after slide digitization and one before the analysis. The first node ensures that the WSI meets the requirements w.r.t. human interpretation and artifacts presented until then (e.g., bubbles, foreign objects, blood, or out-of-focus effect). The second node ensures that the analysis module can be used as intended by being aware of its range of acceptability. It checks that the objects taken as input are coherent w.r.t. what the analysis module has been designed for. For example, a ML algorithm can be self-aware of its range of acceptability by analyzing its training data and by comparing inputs with this range.

All the studies cited for quality control are based on the human perspective to perform QC and answer the question “Can this WSI be analyzed by a pathology expert?”. In other words, they are only addressed to the QC digitization module (digitization node of Fig. 2). None of them are addressed to the QC analysis module (analysis node of Fig. 2): assessing a good WSI digitization quality from human perspective does not necessarily imply a good WSI digitization quality for analysis.

## Limitation of human-based quality assessments in digital pathology

Humans cannot detect all the noise distortions that might interfere with an algorithm performance. Therefore, evaluating WSI digitization quality from a human perspective does not necessarily ensure good quality for automated analysis. To exhibit this, Ullman et al.⁵⁰ introduced the concept of minimal recognizable configurations of images (MIRCs), which are the smallest image crops for which human observers can still predict the correct class. After training deep neural networks on the full images, they tested them on this MIRC dataset and showed that human performance clearly outperformed deep networks. This underlines the fact that modified images may not interfere with human perspective, but with automated analysis. Concept like one-pixel-attack⁴⁷ exhibits this by modifying the value of one pixel in order to completely change the results of an AI algorithm. Even if preventing such an attack is clearly not the main objective of a QC module, it highlights the fact that human and algorithm interpretation of a WSI are clearly not similar. Although the majority of one pixel attacks are against deep learning frameworks, Chen et al.⁷ developed and tested a gradient-based attack against a SVM model working directly in the pixel domain. Applied to WSI, Korpihalkola et al.²¹,²² show that there exists minor one-pixel modification methods that can reverse the automatic diagnosis result of a WSI.

We propose in this article a QC module to assess cytology WSI quality from the algorithm point of view. Assessing QC from an algorithm perspective is already studied in various areas, which we present in the next section.

## Algorithm-centric quality control approaches

Assessing QC from an algorithm perspective is done by preventing a model to process samples that cannot be managed. To do this, works have been done to detect OOD samples before or after applying the algorithm.

In a first approach, outliers are detected by analyzing the confidence in the model's predictions. As deep neural networks tend to produce overconfident predictions,¹² works have been done to separate the confidence scores between in- and out-of-distribution samples. A baseline, based on probabilities from softmax distributions, is introduced by Hendrycks and Gimpel.¹⁷ More effective approaches are then proposed, as in Liang et al.,²⁷ where they use temperature scaling jointly with small perturbations on the input data to better separate the softmax score distributions. Some other OOD methods train a specific deep model to provide, in addition to the prediction, an uncertainty score, using Bayesian neural networks,¹¹ deep ensembles,²⁴ or multi-head convolutional neural networks.³⁰

A second approach is to use data description methods to identify OOD samples. Data description methods make a description of a training set of samples in order to detect if a new sample resemble to this training set. Tax and Duin⁴⁹ proposed a method, inspired by the SVM paradigm, to obtain a spherically shaped boundary with minimum volume around the complete training samples. As performances of such SVM methods are suboptimal on complex and high dimensional datasets, Chalapathy et al.⁶ derived the SVM loss function for a one-class neural network model with the objective of creating a tight envelope around normal data. Lee et al.²⁵ proposed that pre-trained features can be effectively modeled using a class-conditional Gaussian distribution. Using this model, they developed a confidence score to determine how closely a new sample aligns with the fitted distributions.

Similar OOD detection methods have been considered for DP datasets from different DP fields, e.g., in histology²⁸,²⁹, radiology,³³ sonography,²⁰ and ophthalmology.²⁶ To the best of our knowledge, no similar work has yet been done for cytology, which we propose in the following sections.

## Materials and methods

We propose to control the quality of WSI-PPs from the features extracted by the processing pipeline just before analysis (hatched area in Fig. 2). As described in Fig. 1, we assume that the processing pipeline extracts nuclei features. Inspired by data description methods, a WSI-PP is represented by feature distributions, one per nucleus feature, obtained for several WSIs it has produced. The quality of a WSI-PP is evaluated according to several reference WSI-PPs, by comparing their feature distributions with a weighted distance. The weights and the other parameters of the proposed QC method are determined according to a validation process based on grid search. The resulting QC module can trigger quality alerts during both test and validation.

<figure><img src="image_5.png" /><figcaption>**Fig. 2.** Prototypical QC module framework. Modified version of Fig. 13 in Brixtel et al.⁴ The ‘Analysis workflow’ refers to Fig. 1. Hatched area indicates where QC is performed in the present work.</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>Fig. 3. Overview of the dissimilarity measure between two WSI-PPs, w.r.t. to the selection matrix P. WSI-PP representations are obtained via sets of WSIs. These pipelines are then represented via a subset of features depicted by the selection matrix P. For each of these features, the distance function w is performed. Final dissimilarity distance is obtained after analyzing these distances by the aggregator function g.</figcaption></figure>

## WSI-PP representation from the processing pipeline

This section describes the feature extraction steps, and their parameters, used to represent a WSI-PP by nuclei feature distributions from a batch of $n_{wsi}$ WSIs it has produced.

### Nuclei feature extraction

We assume that the processing pipeline is composed of two main blocks: nuclei detection and segmentation (denoted $\phi_{nuc}$), and nuclei feature extraction (denoted $\phi_{feat}$). Given a WSI, or more generally a batch S of $n_{wsi}$ WSIs, $\phi_{nuc}$ extracts a total of $n_{nuc}$ nuclei, i.e., $\frac{n_{nuc}}{n_{wsi}}$ per WSI, and $\phi_{feat}$ extracts $n_{feat}$ scalar features per extracted nucleus. We refer to the application of the two blocks as $\phi(S) = \phi_{feat}(\phi_{nuc}(S))$. In order to simplify the notations, the output nuclei features are encoded by a $n_{nuc} \times n_{feat}$ feature matrix $C = \phi(S)$. The number and nature of the nuclei features we used in the experiments are described later. In this article, for any matrix A, its elements are denoted $A_{ij}$ and its columns $a_j$, i.e., $A = (A_{ij})_{i,j} = [a_j]_j$.

### Nuclei selection

In order to extract representative nuclei, $\phi_{nuc}$ is parameterized by a composition of nuclei selection methods, and by the number $n_{nuc}$ of nuclei to be selected. Three base nuclei selection methods were considered in the experiments:

* The first one, denoted $\phi_{nuc}^{\text{intern}, \tau}$, avoids selecting nuclei located in regions with high nucleus density (clusters). Indeed, the high rate of overlaps between nuclei can lead to segmentation problems. Due to the centrifugation process, clusters are often present at the WSI edges, nuclei are only extracted inside an ellipse centered at the WSI center. The axes of the ellipse are defined as a percentage $\tau$ of the WSI width and height. The values $\tau = 20\%$, 40% were tested in the experiments.

* The second method, denoted $\phi_{nuc}^{\text{distrib}}$, avoids oversampling nuclei in clusters. It extracts evenly distributed nuclei from the WSI.

* The last method, denoted $\phi_{nuc}^{\text{debris}}$, ignores nuclei that are labeled as debris. These preparation artifacts can distort the expressiveness of features and introduce bias. The elements on which a nucleus is considered as debris are exposed in the experiments section.

These base selection methods can be combined, together with the number of extracted nuclei. Several configurations were opposed in the experiments.

## Reference WSI-PPs and nuclei feature selection

Some of the nuclei features considered by $\phi_{feat}$ may be correlated or more relevant in representing WSIs produced by a specific WSI-PP, for

both QC and analysis. Moreover, by eliminating the less relevant features, the interpretability and the demand for computing resources should be improved. To this end, we assume to know $n_{ref}$ reference WSI-PPs, with different preparation characteristics. The processing pipeline can be adapted to a specific WSI-PP i by reducing the number of nuclei features to $n_{sel} \le n_{feat}$. The reduction is encoded by a $n_{feat} \times n_{sel}$ reduction matrix $\mathbf{P}^i$ adapted to the WSI-PP i, i.e., an assignment from $n_{feat}$ features to $n_{sel}$ features: $P_{l,r}^i \in \{0, 1\}$ for all $(l, r)$, $\sum_l P_{l,r}^i = 1$ for all $r$, and $\sum_r P_{l,r}^i \le 1$ for all $l$. The reduction matrices are defined by feature selection from the nuclei feature values obtained for all references. Then, given a batch S of WSIs and a reference WSI-PP i, the nuclei feature matrix adapted to reference i is given by $C = \phi(S) \cdot \mathbf{P}^i$.

### Nuclei feature distributions

In the QC module, a WSI-PP is represented by nuclei feature probability distributions, one per nuclei feature. Given a nuclei feature matrix $C \in R^{n_{nuc} \times n_f}$ with $n_f$ nucleus features (reduced or not), it is mapped to a matrix $F = f(C) \in [0, 1]^{n_{bin} \times n_f}$, where each column $f_l \in [0, 1]^{n_{bin}}$ is a normalized histogram with $n_{bin}$ bins computed by $f$:

$$f(\mathbf{C}) = \begin{cases} \forall l, & \bar{\mathbf{c}}_l = \frac{\mathbf{c}_l - m_l^-}{m_l^+ - m_l^-} \\ \forall l, & \mathbf{h}_l = \text{hist}_{[0,1]}(\bar{\mathbf{c}}_l) \\ \forall l, & s = \sum_l \mathbf{1}^\top \cdot \mathbf{h}_l \\ \forall l, & \mathbf{f}_l = \frac{\mathbf{h}_l}{s} \end{cases} \quad (1)$$

For each feature $l$, the first step normalizes the input feature values to [0,1] w.r.t. reference minimum and maximum feature values $m_l^-$ and $m_l^+$, respectively, fixed later. The second step computes a histogram from the normalized values, obtained by discretizing [0,1] with $n_{bin}$ equal-width bins. The last step ensures that the bin values sum to 1 (discrete probability distribution). We do not assume that all input feature values are in the range $[m_l^-, m_l^+]$, i.e., some normalized values may be outside [0,1]. These eventual out-of-distribution values $(C_{k,l}^* \in [0, 1])$ are not counted by the histogram function, but implicitly taken into account by the last normalization. Also, note that $f(\mathbf{C} \cdot \mathbf{P}) = f(\mathbf{C}) \cdot \mathbf{P}$ holds for any reduction matrix $\mathbf{P}$.

## Quality control and measure

This section describes the QC process. It is based on the knowledge of $n_{ref}$ reference WSI-PPs. Each reference WSI-PP i is represented by a tuple $(\mathbf{R}^i, \mathbf{P}^i, \alpha_i)$ from a reference batch $\mathbf{S}^i$ of $n_{wsi}$ WSIs produced by i, where:
$\mathbf{R}^i = f(\phi(\mathbf{S}^i)) \cdot \mathbf{P}^i$ is the nucleus feature distribution matrix adapted to $i$ via the reduction matrix $\mathbf{P}^i$. The weight $\alpha_i \in [0, 1]$ represents the distance value at which preparation pipelines are considered dissimilar to the reference $i$, w.r.t. a distance function between feature distributions.

## Distance between feature distributions

In the QC module, WSI-PPs are compared w.r.t. their nuclei feature distributions. Let $w: [0, 1]^{n_{\text{bin}}} \times [0, 1]^{n_{\text{bin}}} \rightarrow [0, 1]$ be a distance function between two probability distributions. We define the distance between two distribution matrices $\mathbf{F} \in [0, 1]^{n_{\text{bin}} \times n_f}$ and $\mathbf{F}' \in [0, 1]^{n_{\text{bin}} \times n_f}$ by aggregating the distances obtained by $\omega$ for each feature:

$$d(\mathbf{F}, \mathbf{F}') \equiv g(\{\omega(\mathbf{f}_l, \mathbf{f}'_l), l = 1, \dots, n_f\}), \quad (2)$$

where $g$ aggregates values of the input set, see Fig. 3. We assume that $d(\cdot, \cdot) \in [0, 1]$. We considered in the experiments:

* For $\omega$, the Wasserstein,<fnref n="1" /> ¹⁹ the Bhattacharyya⁴⁰ and the Jensen-Shannon<fnref n="2" /> ¹⁰ distances, denoted $\omega_{\text{wass}}$, $\omega_{\text{bhat}}$, and $\omega_{\text{jsha}}$, respectively.

* For $g$, the max, the mean or the median operators, refereed as $g_{\text{max}}$, $g_{\text{mean}}$, and $g_{\text{med}}$.

## Quality constraints

In order to control the quality of an input preparation pipeline $\mathbf{S}$, we define $\mathbf{S}$ as compatible with a reference $i$ if its nuclei feature distributions are similar to the reference distributions $\mathbf{R}^i$ and dissimilar to the other reference distributions $\mathbf{R}^j$, w.r.t. the nuclei features of each reference. For each reference $j = 1, \dots, n_{\text{ref}}$, the notion of (dis)similarity is controlled by the weight $\alpha_j \in (0, 1)$:

$$d(\mathbf{R}^i, f(\phi(\mathbf{S})) \cdot \mathbf{P}^i) < \alpha_i, \quad (3)$$

$$\forall j \neq i, \quad d(\mathbf{R}^j, f(\phi(\mathbf{S})) \cdot \mathbf{P}^j) \geq \alpha_j. \quad (4)$$

These $n_{\text{ref}}$ constraints can be used to control the quality of a preparation pipeline that should be compatible with a reference preparation pipeline $i$ (prior knowledge), i.e., the preparation should be reconsidered if all constraints are not met, or at least the quality should be examined.

### Quality control process

Given an input batch $\mathbf{S}$ of WSIs and a reference WSI-PP $i$, it is encoded by the distance vector $\mathbf{d} = (d_j(\mathbf{S}))_{j=1, \dots, n_{\text{ref}}}$ to all $n_{\text{ref}}$ references, including $i$, where

$$\forall j, \quad d_j(\mathbf{S}) = \frac{1}{\alpha_j} d(\mathbf{R}^j, f(\phi(\mathbf{S})) \cdot \mathbf{P}^j) - 1 \quad (5)$$

defines a weighted distance from the reference $j$, equivalent to the quality constraints. It is signed, with values in $[-1, \frac{1}{\alpha_j} - 1]$ such that $\mathbf{S}$ is compatible with $i$ iff $d_j(\mathbf{S}) < 0$ for $j = i$, and $d_j(\mathbf{S}) \ge 0$ for all $j \neq i$. Then, the QC process measures both the compatibility and the quality of $\mathbf{S}$. The compatibility measures the normalized number of constraints satisfied for reference $i$:

$$C_i(\mathbf{S}) = \frac{1}{n_{\text{ref}}} [1_{\mathbb{R}_{<0}}(d_i(\mathbf{S})) + \sum_{j \neq i} 1_{\mathbb{R}_{\ge 0}}(d_j(\mathbf{S}))], \quad (6)$$

where $1_{\mathbb{X}}(x) = \begin{cases} 1 & \text{if } x \in \mathcal{X} \\ 0 & \text{if } x \notin \mathcal{X} \end{cases}$ is the indicator function. $\mathbf{S}$ is compatible with $i$ if $C_i(\mathbf{S}) = 1$. The quality measure provides more details by quantifying both

the similarity with the reference $i$ and the dissimilarity with the other references:

$$Q_i(\mathbf{S}) = \beta Q_i^r(\mathbf{S}) + (1 - \beta) Q_i^f(\mathbf{S}), \quad (7)$$

where $Q_i^r(\mathbf{S})$ is the quality measure between $\mathbf{S}$ and the reference on which compatibility is tested, i.e., the reference $i$:

$$Q_i^r(\mathbf{S}) = \max\{0, -d_i(\mathbf{S})\}, \quad (8)$$

and $Q_i^f(\mathbf{S})$ is the quality measure between $\mathbf{S}$ and other references:

$$Q_i^f(\mathbf{S}) = \frac{1}{n_{\text{ref}} - 1} \sum_{j \neq i} \frac{\alpha_j}{1 - \alpha_j} \max\{0, d_j(\mathbf{S})\}, \quad (9)$$

with $\beta \in [0, 1]$ a balancing coefficient. $Q_i(\cdot) \in [0, 1]$, with $Q_i(\mathbf{S}) = 0$ in case no constraint is satisfied. As the distance vector, the compatibility and the quality measure encode different aspects of the quality, the process returns the tuple $(\mathbf{d}, C_i(\mathbf{S}), Q_i(\mathbf{S}))$. It can be directly used to check the quality of the preparation pipeline $i$ or a preparation pipeline assumed to be similar to $i$.

### Maximum quality reference

The QC process can also be used to determine if there is a reference preparation pipeline, and a corresponding processing pipeline, adapted to a given preparation pipeline $\mathbf{S}$. We propose to retain a reference compatible with $\mathbf{S}$ and with maximum quality:

$$\arg \max_{i \in I} Q_i(\mathbf{S}), \quad \text{with } I = \{k \in \{1, \dots, n_{\text{ref}}\} | C_k(\mathbf{S}) = 1\}. \quad (10)$$

When $\mathbf{S}$ is not compatible with any reference ($C_k(\mathbf{S}) < 1$ for all $k$), a new reference should be integrated in the QC module, and validated as described in the next sections.

### Determining reference reduction matrices and weights

This section describes how the nuclei feature space is adapted to each reference WSI-PP, and how the reference weights are defined from these features.

### Reduction matrices

For each reference preparation pipeline $i$, its reduction matrix $\mathbf{P}^i$ is defined by selecting relevant features by a one-versus-all strategy, with regards to the other references. For this, we consider the decision problem (binary classification) which determines, for each nucleus of all references, whether the nucleus comes from the reference preparation $i$ or not. Let $\mathbf{X} \in \mathbb{R}^{n_{\text{ref}} \times n_{\text{nucl}}} \times n_{\text{ref}}$ be the stack of all the nuclei feature matrices $\mathbf{X}^i = \phi(\mathbf{S}^i)$ obtained for all the references, and $\mathbf{y}^i \in \{0, 1\}^{n_{\text{ref}} \times n_{\text{nucl}}}$ be the associated nucleus binary labels, i.e.,

$$\forall k, \quad \mathbf{y}_k^i = \begin{cases} 1 & \text{if nucleus } k \text{ comes from reference } i \\ 0 & \text{else.} \end{cases}$$

The pair $(\tilde{\mathbf{X}}, \mathbf{y}^i)$, with $\tilde{\mathbf{X}}_{k,l} = \frac{\mathbf{x}_{k,l} - \min_{\mathbf{x}_{k,l} < \min_{\mathbf{x}_{k,l} < \min_{\mathbf{x}_{k,l}}}} \mathbf{x}_{k,l}}{\max_{\mathbf{x}_{k,l} < \min_{\mathbf{x}_{k,l} < \min_{\mathbf{x}_{k,l}}}} \mathbf{x}_{k,l}}$, is analyzed by a selection method $\phi_{\text{sel}}$ that returns a $n_{\text{ref}} \times n_{\text{sel}}$ selection matrix $\mathbf{P}^i$, with $n_{\text{sel}} \le n_{\text{ref}}$ features:

$$\mathbf{P}^i = \phi_{\text{sel}}(\tilde{\mathbf{X}}, \mathbf{y}^i, n_{\text{sel}}). \quad (11)$$

For our experiments, the search space for $n_{\text{sel}}$ were restricted to $\{10, 20, 50, 100, \text{all} = n_{\text{feat}}\}$, and two standard selection methods were tested. The first one is based on the chi-squared statistics.<fnref n="3" />, ³⁷ The

<footnote n="1">¹ SciPy implementation: `scipy.stats.wasserstein_distance` of SciPy v1.11.3.</footnote>

<footnote n="2">² SciPy implementation: `scipy.spatial.distance.jensenshannon` of SciPy v1.11.3.</footnote>

<footnote n="3">³ Scikit-learn implementation: `sklearn feature selection chi2` of Scikit-learn v1.2.2.</footnote>
chi-squared test is performed independently for each feature, and the n$_{sel}$ features receiving the lowest p-values are retained. The second selection method is based on minimum redundancy maximum relevance.<fnref n="4" />,⁵³ This method ranks features based on their importance, which is calculated as the quotient between features relevance and redundancy. Relevance is based on F-statistics, whereas redundancy is computed with Pearson correlation. The n$_{sel}$ features receiving the highest importance score are retained. Thus, we considered 10 configurations for $\phi_{sel}$, which are the combination of the two selection methods, denoted $\phi_{sel}^{chi2}$ and $\phi_{sel}^{mmrr}$, with the five possible n$_{sel}$ values.

### Reference weights

The weights $\alpha_i$ involved in the quality constraints (Eqs. 3 and 4) and the quality process are defined by opposing the references to a validation set composed of $n_{val} = n_{ref}$ WSI-PPs. Each validation WSI-PP $i$, defined by a batch $V^i$ of n$_{wsi}$ WSIs, is constrained to have characteristics similar to those of reference $i$. The reference minimum and maximum feature values already introduced, are defined as $m_i^- = \min_{i,k} \{V_{k,l}^i, R_{k,l}^i\}$ and $m_i^+ = \max_{i,k} \{V_{k,l}^i, R_{k,l}^i\}$ for each nucleus feature $l$. By defining the weights by

$$ \forall i, \quad \alpha_i := \min_{j \neq i} d(\mathbf{R}^i, f(\phi(\mathbf{V}^j)) \cdot \mathbf{P}^i), \qquad (12) $$

the quality constraint given by Eq. (4) is necessarily satisfied for all $i$. Contrary to that, the quality constraint given by Eq. (3) may not hold for all $i$. It is captured by the validation accuracy defined in the following. The entire stream to define reference weights is illustrated in Fig. 4.

### Validation process

This section presents the grid search approach we used to find relevant parameter configurations. It is based on a validation set $V = \{V^i\}_i$ of WSI-PPs, as introduced in the previous section, and the accuracy and error metrics described below.

### Parameters

The QC module parameters introduced so far are the number of WSIs $n_{wsi}$, the nuclei selection method(s) $\phi_{nuc}$ and the number of selected nuclei $n_{nuc}$, the distance function $\omega$ and the aggregator $g$, the feature selection method $\phi_{sel}$, and the number of reduced features $n_{sel}$. As $\phi_{nuc}$ methods are random (except for debris), the seed is also a parameter of the QC module. All parameters can take a finite number of values. Let $\Theta^+$ be the set of all possible parameter value combinations $\theta = (\theta_p)_{p=1,...,n_{par}}$, with $n_{par} = 8$ the number of parameters. Note that the number of nuclei features $n_{feat}$ is not a parameter, it is fixed by the processing pipeline.

### Accuracy

The accuracy of the QC module is defined as the sum of compatibilities (from Eq. 6):

$$ A := \sum_{i=1}^{n_{ref}} C_i (V^i) = \frac{1}{n_{ref}} \sum_{i=1}^{n_{ref}} \mathbf{1}_{\mathbb{R}_{<0}} (d_i(S)). \qquad (13) $$

All validation WSI-PPs are compatible with their corresponding reference iff $A = 1$ (i.e., Eq. 3 holds for all $i$). In this case, the QC module is considered as valid. We denote by $A_\theta$ the accuracy obtained for a specific parameter combination $\theta$.

### Error

Several combinations of parameter values can lead to a valid QC module (accuracy of 1). In order to differentiate them, we introduce a error function equivalent to the minimum quality reached by the WSI-PPs in V (Eq. 7). It is defined from normalized distances for more stability. Let $\mathbf{D} \in [0, 1]^{n_{ref} \times n_{ref}}$

be the distance matrix defined by $\mathbf{D}_{i,j} := d(\mathbf{R}^i, f(\phi(\mathbf{V}^j) \cdot \mathbf{P}^i))$, and $\bar{\mathbf{D}}$ its normalized version defined by $\bar{\mathbf{D}}_{i,j} := \frac{D_{i,j} - \mu_j^-}{\mu_j^- - \mu_j^+}$, with $\mu_j^- := \min_i D_{i,j}$ and $\mu_j^+ := \max_i D_{i,j}$. Then, the error made by the QC module is measured by the error function:

$$ \varepsilon := 1 - \min_{i} \min_{j \neq i} \frac{\bar{\mathbf{D}}_{i,j} - \bar{\mathbf{D}}_{i,i}}{\bar{\mathbf{D}}_{i,j} - \bar{\mathbf{D}}_{i,i}} \qquad (14) $$

The difference represents the margin before the quality constraint given by Eq. (3) is invalidated. We denote by $\varepsilon_\theta$ the error obtained for a specific parameter combination $\theta$.

### Parameter selection

We describe the sketch of the grid search process used to find relevant parameters. Its inputs are the reference and validation WSI batches $\mathbf{R}^i$ and $\mathbf{V}^i$, respectively. As the number of combinations is large, in particular many seeds should be used to guarantee stable results, the search space is first reduced to a set $\Theta$ by setting both $n_{wsi}$ and $n_{nuc}$ to reasonable values, and by limiting to a unique seed. All the parameter values are listed in Table 1. Moreover, the impact of $\phi_{nuc}^{debris}$ were analyzed in a separate experiment (reported in Appendix A.2). Then, a set of candidate combinations is determined in three main steps:

1. Select the valid combinations in $\Theta$, i.e., the set $\Theta^* = \{\theta \in \Theta | A_\theta = 1\}$.

2. Select at most $k_1 \le |\Theta^*|$ combinations in $\Theta^*$ that are the smallest according to the error:

$$ \Omega \subseteq \Theta^* \text{ s.t. } |\Omega| \le k_1 \text{ and } \max_{\theta \in \Omega} \varepsilon_\theta \le \min_{\theta \in \Theta \setminus \Omega} \varepsilon_\theta. $$

3. Select at most $k_2 \le |\Omega|$ combinations in $\Omega$ that are largest according to the objective defined by $T(\theta) = (1 - \varepsilon_\theta) + H(\theta)$ for any combination $\theta = (\theta_1, ..., \theta_{n_{par}}) \in \Omega$, where

$$ H(\theta) := \sum_p h^p(\theta_p) $$

is the sum of parameter value frequencies in $\Omega$, and $h^p(t) := |\{\theta \in \Omega | \theta_p = t\}|/|\Omega|$ is the t-value frequency of the p-th parameter. The resulting set is given by

$$ \Omega^- \subseteq \Omega \text{ s.t. } |\Omega^-| \le k_2 \text{ and } \min_{\theta \in \Omega^-} T(\theta) \ge \max_{\theta \in \Omega^-} T(\theta). $$

Then, the set of candidate combinations $\Omega^-$ is refined by examining the stability of the candidate combinations over several $n_{seed}$ new seeds. For each combination $\theta \in \Omega^-$, $n_{seed}$ combinations $\theta^1, ..., \theta^{n_{seed}}$ are created by replacing the initial seed value in $\theta$ by the new ones. The candidate combinations in $\Omega^-$ that remain valid for the $n_{seed}$ new seeds are retained as relevant:

$$ \Omega^* = \{\theta \in \Omega^- | \sum_{s} A_{\theta'} = n_{seed}\}. $$

These relevant combinations are used to determine a final combination, as well as the corresponding reference reduction matrices $\mathbf{P}^i$ and weights $\alpha_i$. For this, accuracies and errors are analyzed for higher $n_{wsi}$ and $n_{nuc}$ values (detailed in the experiments section). The parameter search can only fail in case $\Theta^*$ is empty (step 1) or $\Theta^*$ is empty (refinement step). When this occurs, it is a QC alert that suggests to reconsider the reference and validation sets (e.g., WSI-PPs of poor quality) or the processing pipeline (e.g., unsuitable for some WSI-PPs). In the experiments we carried out, $\Theta^*$ and $\Theta^*$ were never empty, and the sizes of the sets $\Omega$ and $\Omega^-$ always corresponded to the expected $k_1$ and $k_2$ values.

### Materials

To compute reference weights for a set of WSI-PPs, we need access to a WSI-PP corpus as well as a nuclei analysis algorithm.

<footnote n="4">⁴ Implementation: https://github.com/smazzanti/mmr v0.2.8.</footnote>
<figure><img src="image_7.png" /><figcaption>**Fig. 4.** Stages to compute reference weights for a WSI-PP (1) with regards to WSI-PP (2) and (3). (a): Reference representation for WSI-PP (1) is computed. (b): Reduction matrix for WSI-PP (1) is obtained from reference representations of WSI-PPs (1), (2), and (3). (c): Distances between WSI-PPs validation (2*) and (3*) (which are pipelines with same characteristics than (2) and (3)) and WSI-PP reference (1) are computed. Reference weight for WSI-PP (1) is defined from these distances. (d): Distance between WSI-PP validation representation (1*) and WSI-PP reference representation (1) is computed. Constraints from Eqs. (3) and (4) are checked in order to validate the reference weight.</figcaption></figure>

## Analysis pipeline

Cytoprocessor,⁹ a cervical cancer screening system used by cytology laboratories to analyze nuclei, is used as the nuclei extraction algorithm. Analyzing a WSI via Cytoprocessor follows the steps depicted in Fig. 1. After detection and segmentation, 203 features are extracted per nucleus. These features are grouped into six categories: color,³⁸ chromatin,³² gray level co-occurrence matrices,¹⁵ shape,³⁸ and texture.⁴⁸ These feature categories were chosen to optimize the performance of the subsequent analysis, which is carried out by a SVM-based classifier. It classifies nuclei into several classes, including those defined by the Bethesda system,³⁶ in particular abnormal and debris classes. The debris class represents segmented objects not considered as nuclei by the classifier. It is used to define the

nuclei selection method $\phi_{nuc}^{\text{debris}}$ and to test the hypothesis that analyzing non-nuclei and mis-segmented nuclei are detrimental to QC.

## WSI-PPs corpus

We consider seven WSI-PPs, denoted by *pipeline*₁ to *pipeline*₇. Each corpus is composed of 120 WSIs taken from a routine laboratory production that has undergone a Cytoprocessor validation, which consists of randomly taking 60 WSIs to confirm the Cytoprocessor diagnosis by a cytology expert. The main characteristics (preparation, coloration, and scanner type) of the WSI-PPs are summarized in Table 2. Each pipeline produces WSIs with distinct preparation characteristics that can be identified by an expert. Changes in preparation have the most significant impact on the

<table><caption>**Table 1**

List of tested parameters. Cumulative column (Cumul.) indicates whether the parameter can be used cumulatively or not.</caption><thead><tr><th>Step</th><th>Parameter</th><th>Description</th><th>Values</th><th>Cumul.</th></tr></thead><tbody><tr><td rowspan="3">Representation</td><td>n<sub>wsi</sub></td><td>Number of WSIs</td><td>10, 20, 30</td><td>No</td></tr><tr><td>n<sub>nuc</sub></td><td>Number of nucleus</td><td>10<sup>3</sup>, 10<sup>4</sup>, 10<sup>5</sup></td><td>No</td></tr><tr><td>φ<sub>nuc</sub></td><td>Nuclei selection method</td><td>φ<sup>intem,0.2</sup><sub>nuc</sub>, φ<sup>intem,0.4</sup><sub>nuc</sub>, φ<sup>distrib</sup><sub>nuc</sub></td><td>Yes</td></tr><tr><td rowspan="2">Measure</td><td>ω</td><td>Similarity function</td><td>ω<sub>wass</sub>, ω<sub>bhat</sub>, ω<sub>jsha</sub></td><td>No</td></tr><tr><td>g</td><td>Aggregator function</td><td>g<sub>min</sub>, g<sub>max</sub>, g<sub>med</sub></td><td>No</td></tr><tr><td rowspan="2">Feature</td><td>n<sub>sel</sub></td><td>Number of selected features</td><td>10, 20, 50, 100, all = n<sub>feat</sub></td><td>No</td></tr><tr><td>φ<sub>sel</sub></td><td>Feature selection method</td><td>φ<sup>mrmr</sup><sub>sel</sub>, φ<sup>chi2</sup><sub>sel</sub></td><td>No</td></tr></tbody></table>
<table><caption>**Table 2**
WSIs preparation pipelines corpus description.</caption><thead><tr><th>Pipeline</th><th>Preparation</th><th>Coloration</th><th>Scanner</th><th>Dim. (mm²)</th><th>Nb. nuclei (10⁶)</th></tr></thead><tbody><tr><td>pipeline<sub>1</sub></td><td>Surepath</td><td>Surepath</td><td>Pannoramic 1000</td><td>310</td><td>10.4</td></tr><tr><td>pipeline<sub>2</sub></td><td>Thinprep</td><td>Thinprep</td><td>Pannoramic 1000</td><td>587</td><td>18.4</td></tr><tr><td>pipeline<sub>3</sub></td><td>IDC20</td><td>ILSA</td><td>Pannoramic 1000</td><td>631</td><td>36.7</td></tr><tr><td>pipeline<sub>4</sub></td><td>IDC20</td><td>ILSA</td><td>Pannoramic 250</td><td>549</td><td>25.3</td></tr><tr><td>pipeline<sub>5</sub></td><td>IDC20</td><td>ILSA-diapath</td><td>Pannoramic 250</td><td>528</td><td>38</td></tr><tr><td>pipeline<sub>6</sub></td><td>IDC20</td><td>ILSA</td><td>NanoZoomer S360</td><td>448</td><td>22.3</td></tr><tr><td>pipeline<sub>7</sub></td><td>IDC20-C</td><td>ILSA</td><td>Aperio GT 450</td><td>579</td><td>37.9</td></tr></tbody></table>

characteristics of the WSIs, followed by changes in the scanner, and then changes in coloration. For instance, *pipeline*<sub>4</sub> and *pipeline*<sub>5</sub> are more difficult to distinguish because they only differ in their coloration stages, which are very similar. For each WSI-PP, we define the reference batch and the validation batch by randomly splitting the 120 WSIs into two batches of 60 WSIs each.

### Real-world corpus

To confront our QC module to real-world production environments, we dispose of cytology WSIs corpus from three laboratories, produced over periods of several weeks, as summarized in Table 3. For each corpus, no interval of more than four consecutive working days without production of WSIs is present.

### Experiments

In this section, we present the experiments carried out to find a relevant combination of parameter values for the proposed QC module. This set is found empirically by analyzing the results of the validation process. The following detail the application of this validation process to the corpus presented in Table 2.

A relevant combination of parameter values (Table 1) were computed by the validation process, with the validation and reference WSI sets, as defined in the validation process. The parameter values retained at each main stage are detailed in the following.

### Relevant set of combinations Ω*

In the first stage (steps 1–4), the number of WSIs is limited to $n_{\text{wsi}} \in \{10, 20, 30\}$ and the number of nuclei to $n_{\text{nuc}} \in \{10^3, 10^4, 10^5\}$. For steps 1–3, only one seed is used (for $\phi_{\text{nuc}$}). This reduced the number of possible combinations to $|\Theta| = 4374$. Then, the first step computes $|\Theta^*| = 2860$ valid combinations, i.e., they comply with all quality constraints (top accuracy $A = 1$). The second step refines the set of valid combinations by fixing $k_1 = 150$. Taking the 5 % combinations with lowest error values $\mathcal{E}_{\theta}$ creates a set $\Omega$ of 143 combinations which all obtained an error below 0.875. The third step further reduces the number of combinations by selecting those that are best represented in $\Omega$ and minimize $\mathcal{E}$, i.e., maximize $\mathcal{T}$. For each parameter value $t$, its frequency $\hat{p}(t)$ in $\Omega$ is shown in Fig. 6. We can observe that some parameter values are more frequent, i.e., the maximum aggregator $g_{\text{max}}$, the Wasserstein distance $\omega_{\text{wass}}$, the nucleus selection method $\phi_{\text{nuc}}^{\text{intern},0.2} + \phi_{\text{nuc}}^{\text{distrib}}$ and the feature selection method $\phi_{\text{sel}}^{\text{chi2}}$. Moreover, a larger number of WSIs seems valuable. Also, we can observe that the frequencies for the number $n_{\text{sel}}$ of reduced features

is relatively homogeneous above 10. This suggests that $n_{\text{sel}} = 20$ could be sufficient. $\Omega^-$ is obtained by retaining $k_2 = 20$ combinations in $\Omega$ with highest $\mathcal{T}_{\theta}$ values, as shown in Fig. 5. Finally, the refinement step generated a set $\mathcal{E}^*$ of 3 relevant combinations from $\Omega^-$ and 100 seeds (2000 combinations), refereed in Table 4. We observe that the best average results for the error $\mathcal{E}_{\theta}$ are obtained with $\text{comb}_{20}$. Moreover, none of the combinations with $\omega = \omega_{\text{js}}$ was compatible on 100 seeds, which shows that the stability of our quality module with this distance is not sufficient. This can also be seen through the frequency parameter values (Figs. 5 and 6), which are higher for $\omega_{\text{wass}}$, suggesting more stable compatibility results. We retained these 3 relevant combinations for the next step, which investigates the impact of several $n_{\text{wsi}}$ and $n_{\text{nuc}}$ on these combinations.

### Quantity of nuclei and WSIs

In the previous stages, the number of WSIs and nuclei were respectively limited to a maximum of $n_{\text{wsi}} = 30$ and $n_{\text{nuc}} = 10^5$. As larger values could reach better and more stable results, the last stage analyzes their impact while fixing the other parameter values as defined in Table 4. For this, $n_{\text{wsi}}$ was incremented from 10 to 60 with a step of 10, and $n_{\text{nuc}} \in \{10, 10^2, 10^3, 5 \cdot 10^3, 10^4, 5 \cdot 10^4, 10^5, 5 \cdot 10^5\}$ with a non-constant step. As before, the combinations were produced from 100 seeds. For 30 WSIs and $10^3$ nuclei, the accuracy were sufficiently high, with a null standard deviation. For the error, best results were obtained with 60 WSIs, with no gain in stability beyond $10^4$ nucleus, as observed in Fig. 7. So we have kept the latter values. To determine if the results from these three combinations, with $n_{\text{wsi}} = 60$ and $n_{\text{nuc}} = 10^4$, are significantly differentable, we performed a Welch's t-test<sup>5 51</sup> with the null hypothesis that populations have same means, and a standard p-value of 0.05. We choosed a Welch's t-test as the resulting $\mathcal{E}_{\theta}$ distributions followed a normal distribution without respecting homoscedasticity. Normality were checked with the Shapiro-Wilk test,<sup>6 42</sup> and homoscedasticity with the Bartlett's,<sup>7 45</sup> both with a p-value of 0.05. Results of $\text{comb}_{18}$ are significantly differentable of the other two combinations. However, results for $\text{comb}_{20}$ and $\text{comb}_7$ are not differentiable as they obtained a p-value of 0.057. The $\text{comb}_{20}$ and $\text{comb}_7$ only differ in the selection of features. The fact that feature selection has no impact on the results shows that some features are not relevant to our task, and can be removed without loss of performance. However, whatever the combination used between $\text{comb}_{20}$ and $\text{comb}_7$, the relevant set of features is present in it. Therefore, the most relevant feature is always selected via the max aggregator $g_{\text{max}}$. We assume that with other types of aggregator, the selection of features could have a positive impact, as non-relevant features could be used in the aggregation process. We finally retained $\text{comb}_{20}$ as the best combination because it does not use feature selection, thus saving computation time. We denote it $\text{comb}^*$.

<table><caption>**Table 3**
Routine laboratories WSI-PPs corpus description.</caption><thead><tr><th>Laboratory</th><th>Pipeline identifier</th><th>Year</th><th>Production days</th><th>WSI number</th><th>Nuclei number in million</th></tr></thead><tbody><tr><td rowspan="2">L</td><td>pipeline<sub>2</sub></td><td>2020</td><td>33</td><td>3606</td><td>542</td></tr><tr><td>pipeline<sub>2</sub></td><td>2021</td><td>25</td><td>5162</td><td>752</td></tr><tr><td rowspan="2">C</td><td>pipeline<sub>4</sub></td><td>2019</td><td>2</td><td>120</td><td>26</td></tr><tr><td>pipeline<sub>4</sub></td><td>2024</td><td>27</td><td>2220</td><td>723</td></tr><tr><td>M</td><td>pipeline<sub>4</sub></td><td>2021</td><td>43</td><td>3656</td><td>749</td></tr></tbody></table>

### Reference WSI-PP reduction matrices and weights

As the combination $\text{comb}^*$ does not use feature selection, all features were used, which amounts to assigning the identity to the reduction matrices, i.e., $\mathbf{P}^i = \mathbf{I}$ for any reference WSI-PP $i$. Weights $\alpha_i$ associated with

<footnote n="5">5 SciPy implementation: `scipy.stats.ttest_ind` of SciPy v1.11.3.</footnote>

<footnote n="6">6 SciPy implementation: `scipy.stats.shapiro` of SciPy v1.11.3.</footnote>

<footnote n="7">7 SciPy implementation: `scipy.stats.bartlett` of SciPy v1.11.3.</footnote>
<figure><img src="image_8.png" /><figcaption>Fig. 5. Relevant combinations according to $H$ and $1 - \varepsilon$. Circled combinations are those selected for $\Omega^*$. Combinations $comb_{20}$, $comb_7$ and $comb_{18}$ are those retained for $\Omega^*$.</figcaption></figure>

$comb^*$ are listed in Table 6. We observe two main groups with a relatively uniform weight distribution. For $pipeline_4$ and $pipeline_5$, the weights are very small compared to the other pipelines. This indicates that at least one other pipeline instantiated by the corresponding validation batch is very close to the reference according to the distance between distribution matrices. This is consistent with the preparation characteristics of the pipelines (Table 2). Indeed, in terms of these characteristics, $pipeline_4$ and $pipeline_5$ only differ in coloration, so they should be more difficult to distinguish. They are also close to $pipeline_3$ and $pipeline_6$, but with a different scanner. The pipelines 1, 2, and 3 share only the same scanner, and the pipelines 1 and 7 differ in at least two characteristics.

Validation was completed without any quality alerts. In order to identify potential quality issues, the retained combination (Table 5), and associated elements representing references (feature distributions, reduction matrices, weights) were used to test the QC module on large routine corpus from different laboratories.

## Discussion

In this section, we discuss results of the QC module, created from combination $comb^*$ (see Table 5) and reference WSI-PP corpus described in the materials section, on real-world production environments from Table 3. To ensure QC for each laboratory WSI-PP stream, the quality module is executed on batches of 60 WSIs. Batches are contiguous in time and don't share common WSIs.

For each batch, the compatibility measure (Eq. 6) and the quality measure (Eq. 7) with $\beta = 0$ are performed. Despite the fact that the results of these two measures show the same trend, the compatibility measure seems to be too strict, as very few batches are compatible with a pipeline. On the other hand, the quality measure seems to correctly identify the valid pipeline, expect for laboratory C. More details on these results can be found in Table A.2–A.4 of the Appendix. In order to strike a balance between these two measures, we have chosen to present and interpret in this section the results of the quality measure with $\beta = 0.7$.

## Laboratory L

Two streams of WSIs from different timelines in laboratory L are analyzed. The first stream, comprising 3606 WSIs, was produced in 2020, whereas the second stream, consisting of 5162 WSIs, was produced in 2021. Because the WSI-PP of laboratory L shares the same characteristics as $pipeline_1$ (including preparation, coloration, and scanner), it is expected that the reference for $pipeline_1$ yields a higher quality measure compared to the others. As anticipated, the quality measure with respect to $pipeline_1$ is the highest. Additionally, the measure remains stable

## Table 4

Set of relevant combinations of parameter values from $\Omega^*$ that are compatible (i.e. obtained an $A_\theta$ of 1) for all seeds. For last two columns, 'Avg.' means average and 'Std.' means standard deviation.

<table><thead><tr><th>Identifier</th><th>n<sub>wsi</sub></th><th>n<sub>nuc</sub></th><th>φ<sub>nuc</sub></th><th>ω</th><th>g</th><th>φ<sub>sel</sub></th><th>n<sub>sel</sub></th><th>T<sub>θ</sub></th><th>Avg. ε<sub>θ</sub></th><th>Std. ε<sub>θ</sub></th></tr></thead><tbody><tr><td>comb<sub>20</sub></td><td>30</td><td>10<sup>4</sup></td><td>φ<sub>nuc</sub><sup>distrib</sup> + φ<sub>nuc</sub><sup>intern, 0, 2</sup></td><td>ω<sub>wass</sub></td><td>g<sub>max</sub></td><td>None</td><td>all</td><td>1.194</td><td>0.785</td><td>0.055</td></tr><tr><td>comb<sub>7</sub></td><td>30</td><td>10<sup>4</sup></td><td>φ<sub>nuc</sub><sup>distrib</sup> + φ<sub>nuc</sub><sup>intern, 0, 2</sup></td><td>ω<sub>wass</sub></td><td>g<sub>max</sub></td><td>φ<sub>sel</sub><sup>chi2</sup></td><td>100</td><td>1.443</td><td>0.796</td><td>0.053</td></tr><tr><td>comb<sub>18</sub></td><td>30</td><td>10<sup>4</sup></td><td>φ<sub>nuc</sub><sup>intern, 0, 4</sup></td><td>ω<sub>wass</sub></td><td>g<sub>max</sub></td><td>φ<sub>sel</sub><sup>chi2</sup></td><td>20</td><td>1.204</td><td>0.835</td><td>0.054</td></tr></tbody></table>

<figure><img src="image_9.png" /><figcaption>(a) Frequencies for $\phi_{nuc}$ parameters.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>(b) Frequencies for distance $d$ and $\phi_{sel}$ parameters.</figcaption></figure>

<caption>Fig. 6. Frequency of the parameter values. Each style (color and marker) corresponds to a parameter.</caption>
<figure><img src="image_11.png" /><figcaption>Fig. 7. Evolution of error $\mathcal{E}$ for 100 seeds w.r.t. $n_{nuc}$ and $n_{wsi}$.</figcaption></figure>

<table><caption>**Table 5**
Combination comb\* of relevant parameter values retained by the validation process.</caption><thead><tr><th>$n_{wsi}$</th><th>$n_{nuc}$</th><th>$\phi_{nuc}$</th><th>$\omega$</th><th>$g$</th><th>$\phi_{sel}$</th><th>$n_{sel}$</th></tr></thead><tbody><tr><td>60</td><td>$10^4$</td><td>$\phi_{nuc}^{\text{distrib}} + \phi_{nuc}^{\text{intern,0.2}}$</td><td>$\omega_{wass}$</td><td>$g_{max}$</td><td>None</td><td>all</td></tr></tbody></table>

and consistent across all batches and both years, indicating quality stability in the WSI-PP, as illustrated in Fig. 8.

### Laboratory C

The data used to depict laboratory C consists of two streams of WSIs from two different timelines. The first stream comprises 120 WSIs produced in 2019, whereas the second stream consists of 2220 WSIs produced in 2024. As laboratory C's WSI-PP shares the same characteristics as pipeline₄, it is expected that the algorithm for pipeline₄ will yield a higher quality measure compared to the others. Moreover, because pipeline₄ and pipeline₅ are almost identical (see Table 2), the quality measures with respect to pipeline₅ are also anticipated to be higher than the others.

As depicted in Fig. 9, expected quality measures for the 2019 batches are observed, as score for pipeline₄ and pipeline₅ are among the three best measures. However, for 2024 batches, the quality measure for the pipeline₄ and pipeline₅ is no longer among the best measures, alerting us to a potential quality problem.

As our dissimilarity measure employs the max aggregator function $g_{max}$, the final dissimilarity distance is determined by a single feature, allowing for easier interpretation of results. In our case, to differentiate the 2019 and 2024 streams from pipeline₄, we have established that features characterizing green colors are used for all batches. One example of such a feature is green_min, which represents the minimum value of green within a nucleus.

This difference in green can be observed by comparing the distribution of values across random batches from 2019, 2024, pipeline₃, pipeline₄, and pipeline₅. Fig. 10 illustrates this by plotting the continuous probability density curves of the pipelines, obtained using a kernel density estimate.<fnref n="8" />

Additionally, this difference is visually evident, as the nuclei in pipeline₄ appear darker than those from 2019, which in turn are darker than the nuclei from the 2024 pipeline, as shown in Fig. 11.

### Laboratory M

For laboratory M, we have 3656 WSIs produced in the final months of 2021. Similar to the previous laboratory, laboratory M's WSI-PP shares the same characteristics as pipeline₄. Therefore, quality measures with respect to pipeline₄ and pipeline₅ are expected to be maximized, whereas others should be inferior.

In initial experiments, we observed that none of the tested pipelines achieved satisfactory quality measurements (see Fig. A.3 in the Appendix for more details). Given the similarity between pipeline₄ and pipeline₅, we hypothesized that these pipelines come into conflict during their reference weight definition. Consequently, we redefined the weights by merging these two pipelines (see Table A.5 in the Appendix). Note that the parameter values used to define these new weights, specifically those from combination comb\*, are not necessarily optimal because they were determined considering pipeline₄ and pipeline₅ as distinct entities.

With these new weights, we observed that pipeline₄ still did not yield the expected results, whereas pipeline₅ was validated for all batches except those between batch 26 and 33 (see Fig. 12). Fig. 13 shows the features used by the quality measure to differentiate pipeline M from pipeline₂, pipeline₄, and pipeline₅. The features used remain generally consistent, except between batch 26 and 33, where a change in the nuclei red colors appears to be present. This change can be observed by examining the distributions for the feature red_max across different pipelines (see Fig. 14), where a noticeable alteration in the shape of the batch 26 curve is evident compared to previous and subsequent batches. Furthermore, this color deviation is visible upon direct inspection of the nuclei, as depicted in Fig. 15, where the nuclei from batch 26 appear redder (or more violet) than those from the preceding and following batches during the validation process.

To evaluate the relevance of our quality measure with respect to the algorithm's performance, we compared our measure to the confidence levels of the classification algorithm's predictions, specifically using an SVM. We observed a correlation between the average batch classification confidences and the batch quality measures, as shown in Fig. 16. This observation suggests that the deviations detected by our measure are likely detrimental to the algorithm's performance.

<footnote n="8">⁸ SciPy implementation: `scipy.stats.gaussian_kde` of SciPy v1.11.3.</footnote>
<table><caption>**Table 6**
Reference WSI-PP weights $\alpha_i$ associated with combination comb* (Table 5).</caption><thead><tr><th>pipeline<sub>1</sub></th><th>pipeline<sub>2</sub></th><th>pipeline<sub>3</sub></th><th>pipeline<sub>4</sub></th><th>pipeline<sub>5</sub></th><th>pipeline<sub>6</sub></th><th>pipeline<sub>7</sub></th></tr></thead><tbody><tr><td>0.1868</td><td>0.1266</td><td>0.1292</td><td>0.0982</td><td>0.0809</td><td>0.1345</td><td>0.1720</td></tr></tbody></table>

<figure><img src="image_12.png" /><figcaption>**Fig. 8.** Quality measure for each representative pipeline with respect to the batches created for laboratory L. The horizontal dotted black bar is used to separate 2020 and 2021 corpus.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>**Fig. 9.** Quality measure for each representative pipeline with respect to the batches created for laboratory C. The horizontal dotted black bar is used to separate 2019 and 2024 corpus.</figcaption></figure>
<figure><img src="image_14.png" /><figcaption>Fig. 10. Continuous probability density curve for feature green_min. Curves obtained by fitting a kernel density estimate on the observed feature values.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>Fig. 11. Nuclei comparison between laboratory C, *pipeline*<sub>4</sub> and *pipeline*<sub>3</sub>. Nuclei with the median value for feature *green_min* have been chosen for *pipeline*<sub>4</sub> (a), laboratory C pipeline 2019 (b), laboratory C pipeline 2024 (c) and *pipeline*<sub>3</sub> (d).</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>Fig. 12. Quality measure for each representative pipeline with respect to the batches created for laboratory M, when considering pipeline 4 and 5 as same.</figcaption></figure>
<figure><img src="image_17.png" /><figcaption>**Fig. 13.** Features used for the quality measure of laboratory M w.r.t. to different WSI-PP. Features characterizing colors are represented by the respective color (green, red, or blue), whereas others are represented in black. Quality measure is represented via the dotted line. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)</figcaption></figure>

<figure><img src="image_18.png" /><figcaption>**Fig. 14.** Continuous probability density curve for feature *red_max*. Curves obtained by fitting a kernel density estimate on the observed feature values.</figcaption></figure>

<figure><img src="image_19.png" /><figcaption>**Fig. 15.** Set of nuclei images for laboratory M. Nuclei with the median value for feature *red_max* have been chosen for 3 batches of laboratory M pipeline: (a) batch 5; (b) batch 26; (c) batch 43.</figcaption></figure>
<figure><img src="image_20.png" /><figcaption>Fig. 16. Relation between our quality measure and algorithm's confidence.</figcaption></figure>

## Conclusion

In this article, we outlined the need for QC from the algorithmic point of view in DP, and particularly in cytology, where preparation errors can be caused by many variables. To address this, we introduced a QC module to ensure compatibility of a preparation pipeline with diverse algorithms. In contrast to prevalent methods, our approach performs QC from the algorithmic perspective, requires minimal computational resources and offers ease to interpretation.

Through empirical analysis, conducted on seven distinct WSI-PPs, we demonstrated that our method clearly discriminates each pipeline. Additionally, we validated its performance on larger and realistic corpus, detecting quality deviations and highlighting a correlation between the quality measure and the algorithm's confidence.

Whereas our method excels in its simplicity and efficiency, there are avenues for enhancement. Introducing a trainable aggregator function for the dissimilarity measure seems promising, despite it may reduce interpretability. Furthermore, adopting a dissimilarity function capable of measuring

distributions with multiple dimensions could better capture the nuances, probably at the cost of greater computing power. We strongly believe in the necessity of QC from the algorithmic perspective and hope this kind of approach will be extended to other WSI representations, technologies and datasets, thereby advancing the quality and reliability of DP WSIs.

## Funding sources

The work of Paul Barthe is partly supported by the ANRT with Cifre number 2023/0122.

## Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Paul Barthe, Romain Brixtel, Yann Caillot, Benoît Lemoine, Arnaud Renouf, Vianney Thurotte and Ouarda Beniken were employed at Datexim during this study.

## Appendix A. Appendix

### A.1. *Experiments*

#### A.1.1. *Implementation details*

Detection, segmentation, and features calculation were run on an Intel Xeon Silver 4214R Processor (max turbo frequency of 3.5 GHz). Experiments were run on an Intel Core i9-12900K Processor (max turbo frequency of 5.20 GHz).

#### A.1.2. *Calculation time*

The average time required to detect, segment, and compute features to nuclei on one WSI is 12 min. The time required to calculate reference weights with optimal parameters found in our experiments, for a corpus of seven pipelines, is less than 5 min. The time required to test the quality of a pipeline w.r.t. to the seven pipelines contained in our measure, with optimal parameters found in our experiments, is also less than 5 min.

#### A.2. *Validation process with debris class*

The validation process described applied in experiments is repeated with one more parameter value for $\phi_{nuc}: \phi_{nuc}^{debris}$. Has already explained, this selection method removes segmented objects not considered as nuclei by Cytoprocessor classifier. Note that $\phi_{nuc}^{debris}$ is only intended to test the hypothesis that artifacts or failed segmentations are detrimental to QC. Indeed, as the QC module is placed before the analysis, the classification results are inaccessible for Cytoprocessor-based QC in practice.
<figure><img src="image_21.png" /><figcaption>Fig. A.1. Relevance of combinations, including $\phi_{\text{nuc}}^{\text{debris}}$ parameter, according to $\mathcal{H}$ and $\mathcal{E}$. Circled combinations are those selected for $\Omega^{-}$. Combination comb\* is selected after Section 4 validation process.</figcaption></figure>

<caption>**Table A.1**</caption>

Set of relevant combinations of parameter values from $\Omega^*$ that are compatible (i.e. obtained an $A_{\theta}$ of 1) for all seeds. Set obtained with $\phi_{\text{nuc}}^{\text{debris}}$ activated. For last two columns, 'Avg.' means average and 'Std.' means standard deviation.

<table><thead><tr><th>Identifier</th><th>n<sub>wsi</sub></th><th>n<sub>nuc</sub></th><th>φ<sub>nuc</sub></th><th>ω</th><th>g</th><th>φ<sub>sel</sub></th><th>n<sub>sel</sub></th><th>T<sub>θ</sub></th><th>Avg. ε<sub>θ</sub></th><th>Std. ε<sub>θ</sub></th></tr></thead><tbody><tr><td>comb<sup>debris</sup><sub>18</sub></td><td>30</td><td>10<sup>4</sup></td><td>φ<sup>distrib</sup><sub>nuc</sub> + φ<sup>intern, 0, 2</sup><sub>nuc</sub></td><td>ω<sub>wass</sub></td><td>g<sub>avg</sub></td><td>None</td><td>all</td><td>1.215</td><td>0.77</td><td>0.058</td></tr><tr><td>comb<sup>debris</sup><sub>8</sub></td><td>30</td><td>10<sup>4</sup></td><td>φ<sup>distrib</sup><sub>nuc</sub> + φ<sup>intern, 0, 2</sup><sub>nuc</sub></td><td>ω<sub>wass</sub></td><td>g<sub>max</sub></td><td>φ<sup>ch12</sup><sub>sel</sub></td><td>100</td><td>1.282</td><td>0.796</td><td>0.053</td></tr><tr><td>comb<sup>debris</sup><sub>13</sub></td><td>30</td><td>10<sup>4</sup></td><td>φ<sup>distrib</sup><sub>nuc</sub> + φ<sup>intern, 0, 2</sup><sub>nuc</sub> + φ<sup>debris</sup><sub>nuc</sub></td><td>ω<sub>wass</sub></td><td>g<sub>max</sub></td><td>φ<sup>ch12</sup><sub>sel</sub></td><td>100</td><td>1.254</td><td>0.815</td><td>0.034</td></tr><tr><td>comb<sup>debris</sup><sub>19</sub></td><td>30</td><td>10<sup>5</sup></td><td>φ<sup>intern, 0, 2</sup><sub>nuc</sub> + φ<sup>debris</sup><sub>nuc</sub></td><td>ω<sub>wass</sub></td><td>g<sub>max</sub></td><td>φ<sup>ch12</sup><sub>sel</sub></td><td>100</td><td>1.203</td><td>0.817</td><td>0.036</td></tr><tr><td>comb<sup>debris</sup><sub>16</sub></td><td>30</td><td>10<sup>4</sup></td><td>φ<sup>distrib</sup><sub>nuc</sub> + φ<sup>intern, 0, 2</sup><sub>nuc</sub></td><td>ω<sub>wass</sub></td><td>g<sub>max</sub></td><td>φ<sup>ch12</sup><sub>sel</sub></td><td>50</td><td>1.218</td><td>0.819</td><td>0.061</td></tr><tr><td>comb<sup>debris</sup><sub>10</sub></td><td>30</td><td>10<sup>4</sup></td><td>φ<sup>distrib</sup><sub>nuc</sub> + φ<sup>intern, 0, 2</sup><sub>nuc</sub></td><td>ω<sub>wass</sub></td><td>g<sub>max</sub></td><td>φ<sup>ch12</sup><sub>sel</sub></td><td>20</td><td>1.263</td><td>0.819</td><td>0.064</td></tr><tr><td>comb<sup>debris</sup><sub>15</sub></td><td>30</td><td>10<sup>3</sup></td><td>φ<sup>distrib</sup><sub>nuc</sub> + φ<sup>intern, 0, 2</sup><sub>nuc</sub> + φ<sup>debris</sup><sub>nuc</sub></td><td>ω<sub>wass</sub></td><td>g<sub>max</sub></td><td>φ<sup>ch12</sup><sub>sel</sub></td><td>100</td><td>1.219</td><td>0.847</td><td>0.039</td></tr></tbody></table>

### A.2.1. Relevant set of combinations $\Omega^*$

We fixed $n_{wsi} = 30$, $n_{nuc} = 10^5$ and limit the grid search to one seed for step 1 to 3, which reduced the number of possible combinations to $|\Theta| = 8748$. The first step of the validation process computed $|\Theta^*| = 6015$ valid combinations. The second step reduced the set of valid combinations to a set $\Omega$ of 301 combinations, all with an error below 0.866. Then, $\Omega^{-}$ is obtained in the third step by retaining $k_2 = 20$, as shown in Fig. A.1. Finally, the refinement step generated a set $\Omega^*$ of 7 relevant combinations referenced in Table A.1. We observe that the 3 best average results for the error $\mathcal{E}_{\theta}$ were obtained with $comb_{18}^{\text{debris}}$, $comb_{8}^{\text{debris}}$ and $comb_{13}^{\text{debris}}$. As $comb_{8}^{\text{debris}}$ is equivalent to $comb_7$ of experiments section, it was not considered for the next step. Therefore, we retained $comb_{18}^{\text{debris}}$, $comb_{13}^{\text{debris}}$, and $comb_{19}^{\text{debris}}$ to investigate the impact of higher $n_{wsi}$ and $n_{nuc}$.

<figure><img src="image_22.png" /><figcaption>Fig. A.2. Evolution of the error $\mathcal{E}$ for 100 seeds and for debris combinations, w.r.t. $n_{nuc}$ and $n_{wsi}$.</figcaption></figure>
### A.2.2. Quantity of nuclei and WSIs

To analyze the impact of nuclei and WSIs quantity, $n_{wsi}$ and $n_{nuc}$ were incremented as in the experiments section. For 30 WSIs and $10^3$ nuclei, the accuracy were sufficiently high, with a null standard deviation. For the error, best results were obtained with 60 WSIs, with no gain in stability beyond $10^4$ nucleus, see in Fig. A.2. With this quantity of WSIs and nuclei, best results were obtained by $comb_{18}^{debris}$ that reached an average $\mathcal{E}$ of 0.732. To determine if these results, with $n_{wsi} = 60$ and $n_{nuc} = 10^4$, are significantly differentiable, we performed a Welch's t-test as the resulting $\mathcal{E}_\theta$ distributions followed a normal distribution without respecting homoscedasticity. This test was performed with a standard $p$-value of 0.05. The results of $comb_{18}^{debris}$ are significantly different from the other two combinations. However, the results for $comb_{18}^{debris}$ and $comb^*$ are not differentiable because they obtained a $p$-value of 0.117. This suggests that debris objects are useful, or at least not disruptive, to discriminate our WSI-PPs (Table 2).

### A.3. Quality control in routine laboratories

#### A.3.1. Additional details

Tables A.2, A.3, and A.4 detail the number of batches that have validated the quality module constraints, for laboratory L, C, and M. More precisely, they detail the percentage of batches that validate Eq. (3) alone, and the percentage of batches that reached each possible accuracy score from Eq. (13). Fig. A.3 shows the results obtained for laboratory M batches with reference weights obtained by considering reference $pipeline_4$ and $pipeline_5$ as distinct. These weights are specified in Table A.5.

<caption>**Table A.2**</caption>

Percentage of laboratory L batches that verified Eq. (6) (second column) or that obtained each possible accuracy score (third column), w.r.t. to the all pipelines. Lines in bold correspond to pipelines designed for the laboratory.

<table><caption>**Table A.3**</caption><thead><tr><th rowspan="2">Pipeline name</th><th rowspan="2">Eq. 3 verified</th><th colspan="8">Accuracy scores</th></tr><tr><th>0/7</th><th>1/7</th><th>2/7</th><th>3/7</th><th>4/7</th><th>5/7</th><th>6/7</th><th>7/7</th></tr></thead><tbody><tr><td>pipeline<sub>1</sub></td><td>67.35 %</td><td>0</td><td>0 %</td><td>0 %</td><td>1.36 %</td><td>23.13 %</td><td>27.89 %</td><td>47.62 %</td><td>0 %</td></tr><tr><td>pipeline<sub>2</sub></td><td><strong>100 %</strong></td><td><strong>0 %</strong></td><td><strong>0 %</strong></td><td><strong>0 %</strong></td><td><strong>0.68 %</strong></td><td><strong>2.04 %</strong></td><td><strong>17.69 %</strong></td><td><strong>68.71 %</strong></td><td><strong>10.88 %</strong></td></tr><tr><td>pipeline<sub>3</sub></td><td>29.93 %</td><td>0 %</td><td>0 %</td><td>1.36 %</td><td>9.52 %</td><td>49.66 %</td><td>19.73 %</td><td>19.73 %</td><td>0 %</td></tr><tr><td>pipeline<sub>4</sub></td><td>11.56 %</td><td>0 %</td><td>0 %</td><td>0 %</td><td>9.52 %</td><td>70.75 %</td><td>19.73 %</td><td>0 %</td><td>0 %</td></tr><tr><td>pipeline<sub>5</sub></td><td>1.36 %</td><td>0 %</td><td>0 %</td><td>1.36 %</td><td>18.37 %</td><td>69.39 %</td><td>10.88 %</td><td>0 %</td><td>0 %</td></tr><tr><td>pipeline<sub>6</sub></td><td>2.72 %</td><td>0 %</td><td>0.68 %</td><td>1.36 %</td><td>17.01 %</td><td>68.03 %</td><td>11.56 %</td><td>1.36 %</td><td>0 %</td></tr><tr><td>pipeline<sub>7</sub></td><td>0 %</td><td>0 %</td><td>0.68 %</td><td>2.04 %</td><td>17.69 %</td><td>68.71 %</td><td>10.88 %</td><td>0 %</td><td>0 %</td></tr></tbody></table>

Percentage of laboratory C batches that verified eq. 6 (second column) or that obtained each possible accuracy score (third column), w.r.t. to the all pipelines. Lines in bold correspond to pipelines designed for the laboratory.

<table><caption>**Table A.4**</caption><thead><tr><th rowspan="2">Pipeline name</th><th rowspan="2">Eq. 3 verified</th><th colspan="8">Accuracy scores</th></tr><tr><th>0/7</th><th>1/7</th><th>2/7</th><th>3/7</th><th>4/7</th><th>5/7</th><th>6/7</th><th>7/7</th></tr></thead><tbody><tr><td>pipeline<sub>1</sub></td><td>0 %</td><td>0.0 %</td><td>0.0 %</td><td>0.0 %</td><td>0.0 %</td><td>27.03 %</td><td>70.27 %</td><td>2.7 %</td><td>0.0 %</td></tr><tr><td>pipeline<sub>2</sub></td><td>5.13 %</td><td>0.0 %</td><td>0.0 %</td><td>0.0 %</td><td>0.0 %</td><td>27.03 %</td><td>70.27 %</td><td>2.7 %</td><td>0.0 %</td></tr><tr><td>pipeline<sub>3</sub></td><td>92.31 %</td><td>0.0 %</td><td>0.0 %</td><td>0.0 %</td><td>0.0 %</td><td>0.0 %</td><td>0.0 %</td><td>29.73 %</td><td>70.27 %</td></tr><tr><td>pipeline<sub>4</sub></td><td><strong>5.13 %</strong></td><td><strong>0.0 %</strong></td><td><strong>0.0 %</strong></td><td><strong>0.0 %</strong></td><td><strong>0.0 %</strong></td><td><strong>27.03 %</strong></td><td><strong>70.27 %</strong></td><td><strong>2.7 %</strong></td><td><strong>0.0 %</strong></td></tr><tr><td>pipeline<sub>5</sub></td><td><strong>5.13 %</strong></td><td><strong>0.0 %</strong></td><td><strong>0.0 %</strong></td><td><strong>0.0 %</strong></td><td><strong>0.0 %</strong></td><td><strong>27.03 %</strong></td><td><strong>70.27 %</strong></td><td><strong>2.7 %</strong></td><td><strong>0.0 %</strong></td></tr><tr><td>pipeline<sub>6</sub></td><td>0 %</td><td>0.0 %</td><td>0.0 %</td><td>0.0 %</td><td>0.0 %</td><td>27.03 %</td><td>70.27 %</td><td>2.7 %</td><td>0.0 %</td></tr><tr><td>pipeline<sub>7</sub></td><td>25.64 %</td><td>0.0 %</td><td>0.0 %</td><td>0.0 %</td><td>0.0 %</td><td>0.0 %</td><td>70.27 %</td><td>29.73 %</td><td>0.0 %</td></tr></tbody></table>

Percentage of laboratory M batches that verified eq. 6 (second column) or that obtained each possible accuracy score (third column), w.r.t. to the all pipelines. Lines in bold correspond to pipelines designed for the laboratory.

<table><caption>**Table A.5**</caption><thead><tr><th rowspan="2">Pipeline name</th><th rowspan="2">Eq. 3 verified</th><th colspan="8">Accuracy scores</th></tr><tr><th>0/7</th><th>1/7</th><th>2/7</th><th>3/7</th><th>4/7</th><th>5/7</th><th>6/7</th><th>7/7</th></tr></thead><tbody><tr><td>pipeline<sub>1</sub></td><td>0 %</td><td>0.0 %</td><td>0.82 %</td><td>0.0 %</td><td>15.57 %</td><td>45.9 %</td><td>34.43 %</td><td>3.28 %</td><td>0.0 %</td></tr><tr><td>pipeline<sub>2</sub></td><td>0 %</td><td>0.0 %</td><td>0.0 %</td><td>0.0 %</td><td>6.56 %</td><td>39.34 %</td><td>44.26 %</td><td>9.84 %</td><td>0.0 %</td></tr><tr><td>pipeline<sub>3</sub></td><td>8.2 %</td><td>0.0 %</td><td>0.0 %</td><td>17.21 %</td><td>10.66 %</td><td>26.23 %</td><td>36.89 %</td><td>5.74 %</td><td>3.28 %</td></tr><tr><td>pipeline<sub>4</sub></td><td><strong>31.15 %</strong></td><td><strong>0.0 %</strong></td><td><strong>0.0 %</strong></td><td><strong>5.74 %</strong></td><td><strong>18.03 %</strong></td><td><strong>22.95 %</strong></td><td><strong>32.79 %</strong></td><td><strong>20.49 %</strong></td><td><strong>0.0 %</strong></td></tr><tr><td>pipeline<sub>5</sub></td><td><strong>86.89 %</strong></td><td><strong>0.0 %</strong></td><td><strong>0.0 %</strong></td><td><strong>0.82 %</strong></td><td><strong>9.02 %</strong></td><td><strong>27.05 %</strong></td><td><strong>14.75 %</strong></td><td><strong>21.31 %</strong></td><td><strong>27.05 %</strong></td></tr><tr><td>pipeline<sub>6</sub></td><td>0 %</td><td>0.0 %</td><td>5.74 %</td><td>22.13 %</td><td>14.75 %</td><td>22.13 %</td><td>30.33 %</td><td>4.92 %</td><td>0.0 %</td></tr><tr><td>pipeline<sub>7</sub></td><td>0 %</td><td>0.0 %</td><td>6.56 %</td><td>22.95 %</td><td>13.93 %</td><td>22.95 %</td><td>30.33 %</td><td>3.28 %</td><td>0.0 %</td></tr></tbody></table>

Reference WSI-PP weights $\alpha_i$ associated with combination $comb^*$ (Table 5) without considering $pipeline_4$ and $pipeline_5$ as distinct WSI-PPs.

<table><thead><tr><th>pipeline<sub>1</sub></th><th>pipeline<sub>2</sub></th><th>pipeline<sub>3</sub></th><th>pipeline<sub>4</sub></th><th>pipeline<sub>5</sub></th><th>pipeline<sub>6</sub></th><th>pipeline<sub>7</sub></th></tr></thead><tbody><tr><td>0.1859</td><td>0.1263</td><td>0.1297</td><td>0.1118</td><td>0.1271</td><td>0.1337</td><td>0.1730</td></tr></tbody></table>
<figure><img src="image_23.png" /><figcaption>Fig. A.3. Quality measure for each pipeline of our metric with respect to the batches created for laboratory M.</figcaption></figure>

## References

1. Abels E, Pantanowitz L, Aeffner F, et al. Computational pathology definitions, best practices, and recommendations for regulatory guidance: a white paper from the digital pathology association. J Pathol 2019;249(3):286-294. https://doi.org/10.1002/path.5331.

2. Ameisen D, Deroulers C, Perrier V, et al. Stack or trash? Quality assessment of virtual slides. Diagn Pathol 2013;8:S23. https://doi.org/10.1186/1746-1596-8-S1-S23.

3. Baxi V, Edwards R, Montalto M, Saha S. Digital pathology and artificial intelligence in translational medicine and clinical practice. Mod Pathol 2022;35(1):23-32. https://doi.org/10.1038/s41379-021-00919-2.

4. Brixtel R, Bougleux S, Lézoray O, et al. Whole slide image quality in digital pathology: review and perspectives. IEEE Access 2022;10:131005-131035. https://doi.org/10.1109/ACCESS.2022.3227437.

5. Campanella G, Rajanna AR, Corsale L, Schüffler PJ, Yagi Y, Fuchs TJ. Towards machine learned quality control: a benchmark for sharpness quantification in digital pathology. Comput Med Imaging Graph 2018;65:142-151. https://doi.org/10.1016/j.compmedimag.2017.09.001.

6. Chalapathy R, Menon AK, Chawla S. Anomaly detection using one-class neural networks 2018. https://doi.org/10.48550/arXiv.1802.06360.

7. Chen Z, Tondi B, Li X, Ni R, Zhao Y, Barni M. A gradient-based pixel-domain attack against SVM detection of global image manipulations. 2017 IEEE Workshop on Information Forensics and Security (WIFS). IEEE; 2017. p. 1-6. https://doi.org/10.1109/WIFS.2017.8267668.

8. Cheng JY, Abel JT, Balis UG, McClintock DS, Pantanowitz L. Challenges in the development, deployment, and regulation of artificial intelligence in anatomic pathology. Am J Pathol 2021;191(10):1684-1692. https://doi.org/10.1016/j.aijpath.2020.10.018.

9. Crowell EF, Bazin C, Saunier F, et al. CytoProcessor™; a new cervical cancer screening system for remote diagnosis. Acta Cytol 2019;63(3):215-223. https://doi.org/10.1159/000497111.

10. Endres D, Schindelin J. A new metric for probability distributions. IEEE Trans Inf Theory 2003;49(7):1858-1860. https://doi.org/10.1109/TIT.2003.813506.

11. Gal Y, Ghahramani Z. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. International Conference on Machine Learning. PMLR; 2016. p. 1050-1059. https://doi.org/10.48550/arXiv.1506.02142.

12. Guo C, Pleiss G, Sun Y, Weinberger KQ. On calibration of modern neural networks. International Conference on Machine Learning. PMLR; 2017. p. 1321-1330. https://doi.org/10.48550/arXiv.1706.04599.

13. Haghighat M, Browning L, Sirinukunwattana K, et al. PathProfiler: Automated quality assessment of retrospective histopathology whole slide image cohorts by artificial intelligence - a case study for prostate cancer research. Sep 2021. https://doi.org/10.1101/2021.09.24. 21263762.

14. Haghighat M, Browning L, Sirinukunwattana K, et al. Automated quality assessment of large digitised histology cohorts by artificial intelligence. Sci Rep 2022;12(1):5002. https://doi.org/10.1038/s41598-022-08351-5.

15. Haralick RM, Shanmugam K, Dinstein IH. Textural features for image classification. IEEE Trans Syst Man Cybernet 1973;6:610-621. https://doi.org/10.1109/TSMC.1973.4309314.

16. Hashimoto N, Bautista PA, Yamaguchi M, Ohyama N, Yagi Y. Referenceless image quality evaluation for whole slide imaging. J Pathol Inform 2012;3(1):9. https://doi.org/10.4103/2153-3539.93891.

17. Hendrycks D, Gimpel K. A baseline for detecting misclassified and out-of-distribution examples in neural networks. 2016. https://doi.org/10.48550/arXiv.1610.02136.

18. Janowczyk A, Zuo R, Gilmore H, Feldman M, Madabhushi A. HistoQC: an open-source quality control tool for digital pathology slides. JCO Clin Cancer Inform 2019;3:1-7. https://doi.org/10.1200/CCI.18.00157.

19. Kantorovich LV. Mathematical methods of organizing and planning production. Manag Sci 1960;6(4):366-422. https://doi.org/10.1287/mnsc.6.4.366.

20. Karimi D, Zeng Q, Mathur P, et al. Accurate and robust deep learning-based segmentation of the prostate clinical target volume in ultrasound images. Med Image Anal 2019;57: 186-196. https://doi.org/10.1016/j.media.2019.07.005.

21. Korpihalkola J, Sipola T, Kokkonen T. Color-optimized one-pixel attack against digital pathology images. 2021 29th Conference of Open Innovations Association (FRUCT). IEEE; 2021. p. 206-213. https://doi.org/10.23919/FRUCT52173.2021.9435562.

22. Korpihalkola J, Sipola T, Puuska S, Kokkonen T. One-pixel attack deceives computer-assisted diagnosis of cancer. 2021 4th International Conference on Signal Processing and Machine Learning; 2021. p. 100-106. https://doi.org/10.1145/3483207.3483224.

23. Lahrmann B, Valous NA, Eisenmann U, Wentzensen N, Grabe N. Semantic focusing allows fully automated single-layer slide scanning of cervical cytology slides. PLoS One 2013;8(4), e61441. https://doi.org/10.1371/journal.pone.0061441.

24. Lakshminarayanan B, Pritzel A, Blundell C. Simple and scalable predictive uncertainty estimation using deep ensembles. Adv Neural Inf Proces Syst 2017;30. https://doi.org/10.48550/arXiv.1612.01474.

25. Lee K, Lee K, Lee H, Shin J. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. 2018. https://doi.org/10.48550/arXiv.1807.03888.

26. Leibig C, Allken V, Ayhan MS, Berens P, Wahl S. Leveraging uncertainty information from deep neural networks for disease detection. Sci Rep 2017;7(1):17816. https://doi.org/10.1038/s41598-017-17876-z.

27. Liang S, Li Y, Srikant R. Enhancing the reliability of out-of-distribution image detection in neural networks. 2018. https://doi.org/10.48550/arXiv.1706.02690.

28. Linmans J, Elfwing S, Van Der Laak J, Litjens G. Predictive uncertainty estimation for out-of-distribution detection in digital pathology. Med Image Anal 2023;83, 102655. https://doi.org/10.1016/j.media.2022.102655.

29. Linmans J, Raya G, Van Der Laak J, Litjens G. Diffusion models for out-of-distribution detection in digital pathology. Med Image Anal 2024;93, 103088. https://doi.org/10.1016/j.media.2024.103088.

30. Linmans J, van der Laak J, Litjens G. Efficient out-of-distribution detection in digital pathology using multi-head convolutional neural networks. MIDL; 2020. p. 465-478 https://proceedings.mlr.press/v121/linmans20a.html.

31. Mach AJ, Adeyiga OB, Di Carlo D. Microfluidic sample preparation for diagnostic cytology. Lab Chip 2013;13(6):1011. https://doi.org/10.1039/c2lc41104k.

32. Mehnert AJH. Image Analysis for the Study of Chromatin Distribution in Cell Nuclei. PhD thesis, Brisbane, Australia: University of Queensland. 2003. https://doi.org/10.14264/209764.

33. Nair T, Precup D, Arnold DL, Arbel T. Exploring uncertainty measures in deep networks for multiple sclerosis lesion detection and segmentation. 2018. https://doi.org/10.48550/arXiv.1808.01200.

34. Nam S, Chong Y, Jung CK, et al. Introduction to digital pathology and computer-aided pathology. J Pathol Transl Med 2020;54(2):125-134. https://doi.org/10.4132/jptm.2019.12.31.

35. Ogura M, Kiyuna T, Yoshida H. Impact of blurs on machine-learning aided digital pathology image analysis. Artif Intel Cancer 2020;1(1):31-38. https://doi.org/10.35713/aic.v1.i1.31.

36. Pangarkar MA. The Bethesda system for reporting cervical cytology. Cytojournal 2022;19. https://doi.org/10.25259/CMAS.03_07_2021.

37. Pearson K, X. On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Lond Edbin Dublin Philos Magaz J Sci 1900;50(302):157-175. https://doi.org/10.1080/14786440009463897.

38. Rodenacker K, Bengtsson E. A feature set for cytometry on digitized microscopic images. Anal Cell Pathol 2003;25(1):1-36. https://doi.org/10.1155/2003/548678.

39. Sakamoto T, Furukawa T, Lami K, et al. A narrative review of digital pathology and artificial intelligence: focusing on lung cancer. Translat Lung Cancer Res 2020;9(5):2255-2276. https://doi.org/10.21037/tlcr-20-591.
40. Sen PK. Anil kumar bhattacharyya (1915-1996): a reverent remembrance. Calc Stat Assoc Bull 1996;46(3):151-158. https://doi.org/10.1177/0008068319960301.

41. Shakhawat HM, Nakamura T, Kimura F, Yagi Y, Yamaguchi M. Automatic quality evaluation of whole slide images for the practical use of whole slide imaging scanner. ITE Trans Media Technol Appl 2020;8(4):252-268. https://doi.org/10.3169/mta.8.252.

42. Shapiro SS, Wilk MB. An analysis of variance test for normality (complete samples). Biometrika 1965;52(3-4):591-611. https://doi.org/10.2307/2333709.

43. Smit G, Ciompi F, Cigéhn M, Bodén A, van der Laak J, Mercan C. Quality control of whole-slide images through multi-class semantic segmentation of artifacts. Medical Imaging with Deep Learning; 2021https://api.semanticscholar.org/CorpusID:236922091.

44. Smith B, Hermsen M, Lesser E, Ravichandar D, Kremers W. Developing image analysis pipelines of whole-slide images: pre- and post-processing. J Clin Translat Sci 2021;5(1), e38. https://doi.org/10.1017/cts.2020.531.

45. Snedecor GW, Cochran WG. *Statistical Methods. 8th ed.* Iowa State University Press. 1989.

46. Song Z, Zou S, Zhou W, et al. Clinically applicable histopathological diagnosis system for gastric cancer detection using deep learning. Nat Commun 2020;11(1):4294. https://doi.org/10.1038/s41467-020-18147-8.

47. Su J, Vargas DV, Kouichi S. One pixel attack for fooling deep neural networks. IEEE Trans Evol Comput 2019;23(5):828-841. https://doi.org/10.1109/TEVC.2019.2890858.

48. Tamura H, Mori S, Yamawaki T. Textural features corresponding to visual perception. IEEE Trans Syst Man Cybernet 1978;6:460-473. https://doi.org/10.1109/TSMC.1978.4309999.

49. Tax DM, Duin RP. Support vector data description. Mach Learn 2004;54(1):45-66. https://doi.org/10.1023/B:MACH.0000008084.60811.49.

50. Ullman S, Assif L, Fetaya E, Harari D. Atoms of recognition in human and computer vision. Proc Natl Acad Sci 2016;113(10):2744-2749. https://doi.org/10.1073/pnas.1513198113.

51. Welch BL. The generalization of student's problem when several different population variances are involved. Biometrika 1947;34(1-2):28-35. https://doi.org/10.2307/2332510.

52. Zhang T, Carvajal J, Smith DF, et al. SlideNet: Fast and accurate slide quality assessment based on deep neural networks. 2018. https://doi.org/10.48550/arXiv.1803.07240.

53. Zhao Z, Anand R, Wang M. Maximum relevance and minimum redundancy feature selection methods for a marketing machine learning platform. 2019. https://doi.org/10.1109/DSAA.2019.00059.