SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Federico Matteucci<sup>1</sup>, Vadim Arzamasov<sup>1</sup>, and Klemens Böhm<sup>1</sup>
$^{1}$ Karlsruhe Institute of Technology {federico.matteucci, vadim.arzamasov, klemens.boehm}@kit.edu
Categorical encoders transform categorical features into numerical representations that are indispensable for a wide range of machine learning models. Existing encoder benchmark studies lack generalizability because of their limited choice of 1. encoders, 2. experimental factors, and 3. datasets. Additionally, inconsistencies arise from the adoption of varying aggregation strategies. This paper is the most comprehensive benchmark of categorical encoders to date, including an extensive evaluation of 32 configurations of encoders from diverse families, with 48 combinations of experimental factors, and on 50 datasets. The study shows the profound influence of dataset selection, experimental factors, and aggregation strategies on the benchmark's conclusions — aspects disregarded in previous encoder benchmarks. Our code is available at https://github.com/DrCohomology/EncoderBenchmarking.
Learning from categorical data poses additional challenges compared to numerical data, due to a lack of inherent structure such as order, distance, or kernel. The conventional solution is to transform categorical attributes into a numerical form, i.e., encode them, before feeding them to a downstream Machine Learning (ML) model. Various encoders have been proposed, followed by several benchmark studies. However, their combined results remain inconclusive, as we now describe.
Many factors impact the generalizability [26] of a benchmark of encoders, including: 1. the compared encoders, 2. the number of datasets, 3. the quality metrics, 4. the ML models used, and 5. the tuning strategy. We also hypothesize that 6. the aggregation strategy used to summarize the results of multiple experiments may affect the conclusions of a study. Existing encoder benchmarks, reviewed in Section 2, only partially control for these factors. First, none of these studies uses more than 15 datasets of a given type (regression or classification). Second, despite these studies collectively covering a substantial number of encoders, they often focus on specific encoder families, resulting in comparison gaps between the best encoders. For instance, the best-performing encoders from [28] (Cross-Validated GLMM) and [44] (Mean-Target) have not been studied together yet. Third, the results of existing studies are often not comparable due to variations in the selected quality metrics. For instance, [28] measures quality with ROC AUC, [4] with average precision, and [41] with accuracy. Fourth, existing studies tune ML models in different ways, yielding incompatible evaluations. For instance, [28, 41] do not tune, while [4, 5, 8, 44] tune but do not specify if they tune the ML model on encoded data or if they tune the entire ML pipeline. Last, no benchmark study of categorical encoders explores the impact of aggregation strategies, which is substantial according to our experiments. For instance, [5] ranks the encoders by average ranking across all datasets, while [28] computes the median ranking with Kemeny-Young aggregation [46].
This study offers a taxonomy and a comprehensive experimental comparison of encoders for binary classification, taking into account the factors just mentioned. In particular, we consider: 1. 32 encoder configurations, including all of the best-performing ones from the literature and three novel encoders; 2. 50 datasets for binary classification; 3. four quality metrics; 4. five widely used ML models; 5. three tuning strategies; 6. 10 aggregation strategies gathered from existing categorical encoder benchmarks and from benchmarking methodology studies [27, 9]. This allows us to provide novel insights into the sensitivity of experimental results to experimental factors. In particular, we demonstrate how replicability [26] may not be ensured even for studies conducted on up to 25 datasets. For those combinations of experimental factors that show reproducible results, we isolate and recommended the best encoders.
Paper outline: Section 2 reviews existing works, Section 3 presents a taxonomy of encoder families, Section 4 describes the experimental setup, and Section 5 features the results.
Table 1: Related work on categorical encoders for binary classification.
# Binary classification datasets	Ours	[28]	[4]	[8]	[5]	[44]	[41]
50	10	5	3	2	2	6
# ML models	5	5	1	4	2	1	5
Encoder family	Identifier	✓	✓	✓	✓	✓	✓	✓
Frequency-based	✓	✓
Contrast	✓	✓
Similarity	✓	✓	✓
Simple target	✓	✓	✓	✓
Binning	✓	✓
Smoothing	✓	✓	✓	✓
Data-constraining	✓	✓	✓
Quality metric	Precision-recall based	✓	✓	✓	✓
Balanced accuracy	✓	✓	✓
Accuracy	✓	✓	✓	✓
Tuning strategy	Full pipeline tuning	✓	?	✓	✓*
Model tuning	✓	✓
No tuning	✓	✓	✓
Aggregation strategy	Heuristic	✓	✓
Friedman-Nemenyi	✓	✓	✓
Kemeny-Young	✓	✓
2 Related work
**Benchmarks of encoders.** We focus on binary classification tasks, as they offer a wider range of compatible encoders; indeed, we could conduct a deeper replicability analysis while maintaining the computation feasible. Table 1 summarizes the related work. The other benchmarks often consider few datasets and either do not tune the ML model or do not describe the tuning procedure. This limits their applicability and generalizability. Additionally, there are substantial differences in the experimental settings across articles, including the encoders considered, quality metrics employed, and aggregation strategies used to interpret results. Hence, the comparability of these findings is limited. For instance, [28] recommends a data-constraining encoder, [41] both data-constraining and contrast encoders, [5, 4] similarity encoders, [8] an identifier encoder, and [44] a simple target encoder. Other benchmarks of encoders are [36], which focuses on regression tasks and faces similar issues, and [30, 14, 20], that use only a single dataset.
Analysis of benchmarks. When designing our benchmark, we adhered to the best practices discussed in the literature on benchmark design and analysis. In particular, [27] studies how choices of experimental factors impact the experimental results and advocates for benchmarks that consider a large variety of factors. Similarly, [9] suggests guidelines to mitigate the inconsistencies in the choices of data and evaluation metric. Finally, [2] proposes a methodology to account for variance in the design choices (randomization of sources of variation) and post-processing of the experimental results (significant and meaningful improvements).
3 Taxonomy of encoders
This section presents the essential terminology and discusses the considered encoders and their corresponding families. Appendix 7.1 provides formal and detailed descriptions of the encoders.
3.1 Notation and terminology
Consider a tabular dataset with target  $y$  taking values in  $\{0,1\}$ , and let  $\mathbf{A}$  be one of its attributes (columns).  $\mathbf{A}$  is categorical if it represents qualitative properties and takes values in a finite domain  $\Omega_A$ . Each  $\omega \in \Omega_A$  is a level of  $\mathbf{A}$ . Categorical attributes do not support arithmetic operations like addition or multiplication, and their comparison is not based on arithmetic relations. An encoder  $E$  replaces a categorical attribute  $\mathbf{A}$  with a set of numerical attributes,  $E(\mathbf{A})$ . We write  $E(\Omega_A)$  to indicate the domain of  $E(\mathbf{A})$ . Encoders may encode different levels in  $\mathbf{A}$  in the same way, or encode in different ways different occurrences in the dataset of the same level. Encoders are either supervised or unsupervised: Supervised encoders require a target column, while unsupervised encoders solely rely on  $\mathbf{A}$ . In what follows,  $\mathbf{A}$  always denotes the categorical attribute to be encoded.
3.2 Unsupervised encoders
Identifier encoders assign a unique vector identifier to each level. The most recognized encoder is One-Hot (OH), the default encoder in most machine learning pipelines [11, 15]. One-Hot is both space-inefficient and ineffective [28, 4, 5]. Alternatives include Ordinal (Ord), which assigns a unique consecutive identifier to each level, and Binary (Bin), which splits the base-2 representation of Ord(A) into its digits.
Frequency-based encoders replace levels with some function of their frequency in the dataset. We use Count, which relies on absolute frequencies [28].
Contrast encoders encode levels into  $(L - 1)$ -dimensional vectors so that the encodings of all levels sum up to  $(0, \dots, 0)$  [41]. A constant intercept term, 1, is usually appended to the encoding of each level. Contrast encoders encode levels such that their coefficients represent the level's effect contrasted against a reference value. A common example is Sum, which contrasts against the target's average value.
Similarity encoders treat  $\omega \in \Omega_A$  as strings and map them into a numeric space taking their similarity into account [5, 4]. These encoders are particularly useful for handling "dirty" categorical datasets that may contain typos and redundancies. One example is Min-Hash (MH), which decomposes each level into a set of  $n$ -grams, sequences of  $n$  consecutive letters, and encodes to preserve the Jaccard similarity of the decompositions.
3.3 Supervised encoders
Simple target encoders encode levels with a function of the target. Prime examples are Mean-Target (MT) [7], which encodes with the conditional average  $y$  given  $\mathbf{A}$ , and Weight of Evidence (WoE) [39], which encodes with the logit of MT(A). As Mean-Target can lead to severe overfitting [28, 31], it may benefit from regularization. The following families of encoders are regularization for Mean-Target.
We propose Binning encoders, that regularize MT by partitioning either  $\Omega_A$  or  $\mathrm{MT}(\Omega_A)$  into bins. Pre-Binned MT (PBMT) partitions  $\Omega_A$  to maximize the number of bins such that each bin's relative frequency exceeds a specified threshold, then encodes the binned attribute with MT. Discretized MT (DMT) partitions  $\mathrm{MT}(\Omega_A)$  into intervals of equal length, then encodes each level with the lower bound of the interval in which its MT encoding falls.
Smoothing encoders blend  $\mathrm{MT}(\Omega_A)$  with the overall average target. Notable examples are Mean-Estimate (ME) [22], which uses a weighted average of the two, and Generalized Linear Mixed Model encoder (GLMM) [28], which encodes with the coefficients of a generalized linear mixed model fitted on the data.
Data-constraining encoders regularize  $\mathrm{MT}(\mathbf{A})$  by restricting the amount of data used to encode each occurrence of a level in the dataset. CatBoost (CB) [31] first randomly permutes the dataset's rows, then maps each occurrence of a level  $\omega$  to the average target of its previous occurrences. Cross-Validated MT (CVMT) [28] splits the dataset into folds of equal size, then encodes each fold with an
MT trained on the other folds. We propose the BlowUp variant of CVMT, BUMT, which trains an MT on each fold and uses them to encode the whole dataset. Related variants are Cross-Validated GLMM (CVGLMM) [28] and its BlowUp version (BUGLMM).
4 Experimental design
As there is no intrinsic measure of an encoder's quality, we proxy the latter with the quality of an ML model trained on encoded data. This procedure is in line with the current literature on the topic, discussed in Section 2. Each experiment thus consists of the following steps. First, we fix a combination of factors: a dataset, an ML model, a quality metric, and a tuning strategy. Then, we partition the dataset using a 5-fold stratified cross-validation and pre-process the training folds by:
- imputing missing values with median for numerical and mode for categorical attributes;
- scaling the numerical attributes;
- encoding the categorical attributes.
If tuning is to be applied, we fine-tune the pipeline with nested cross-validation and output the average performance over the outer test folds. We used standard scikit-learn [29] procedures for scaling and missing values imputation.
We conducted experiments using Python 3.8 on an AMD EPYX 7551 machine with 32 cores and 128 GB RAM. We limit each evaluation to 100 minutes to handle the extensive workload. As described in Appendix 7.3.1, out of the 64000 cross-validated evaluations, 61812 finished on time without throwing errors. For the sensitivity, replicability, and encoder comparison analysis, we ignored the missing evaluations. We did so 1. since there is no clearly superior imputation method, and 2. to avoid introducing unnecessary variability in the analysis. Our preliminary experiments confirm that imputing the small number of missing evaluations does not significantly impact our analysis.
In what follows, we describe the datasets, ML models, quality metrics, and tuning strategies we use in our experiments. Then, we outline the different aggregation strategies. Appendix 7.2 provides further details about datasets and aggregation strategies.
4.1 Encoders
We used the category_encoders<sup>1</sup> implementations of Bin, CB, Count, Ord, OH, Sum, and WoE. We sourced MH from the authors' implementation [4, 5].<sup>2</sup> We implemented DMT, GLMM, ME, MT, PBMT, CVMT, BUMT, CVGLMM, and BUGLMM. We also added a baseline encoder, Drop, which encodes every level with 1. For DMT, we experimented with the number of bins:  $\{2, 5, 10\}$ , for ME, with the regularization strength:  $\{0.1, 1, 10\}$ , for PBMT, with the minimum frequency:  $\{0.001, 0.01, 0.1\}$ , and for cross-validated encoders, such as CVMT, with the number of folds:  $\{2, 5, 10\}$ . We display hyperparameter values with subscripts, e.g.,  $\mathrm{CV}_2\mathrm{MT}$ .
4.2 Datasets
We used binary classification datasets. This allows us to conduct in-depth analysis using the same ML models and quality metrics. Additionally, certain supervised encoders, e.g., WoE, are specifically designed for binary classification tasks. We chose 50 datasets with categorical attributes from OpenML [42], including the suitable ones from the related work.
4.3 ML models
We experimented with diverse ML models that process data in different ways: decision trees (DT) and boosted trees (LGBM) exploit orderings, support vector machines (SVM) use kernels, k-nearest neighbors (k-NN) relies on distances, and logistic regression (LogReg) is a "pseudo-linear" model. The LGBM implementation we used is from the LightGBM module, while the other models' implementations are from scikit-learn. Table 2 compares our model choices with related work. We
<https://contrib.scikit-learn.org/category_encoders/
$^{2}$ https://dirty-cat.github.io/stable/
<sup>3</sup>https://lightgbm.readthedocs.io/en/v3.3.5/
Table 2: ML models used in related studies.
Ours	[28]	[4]	[8]	[5]	[44]	[41]
Model family	Tree ensembles	✓	✓	✓	✓	✓	✓
Linear	✓	✓	✓	✓
SVM	✓	✓	✓	✓
k-NN	✓	✓
DT	✓	✓	✓	✓
Neural	✓	✓
Naïve Bayes	✓
excluded neural models due to their inferior performance on tabular data [15] and the absence of a recommended architecture. We also did not use Naive Bayes due to its lack of popularity.
4.4 Quality metrics and tuning strategies
We assessed an encoder's quality by evaluating an ML model trained on the encoded data. We use four quality metrics: balanced accuracy (BAcc), F1-score (F1), accuracy (Acc), and Area Under the ROC Curve (AUC). We compared three tuning strategies:
- no tuning;
- model tuning: the entire training set is pre-processed before tuning the model;
- full tuning: the entire pipeline is tuned on the training set, with each training fold of the nested cross-validation pre-processed independently.
We used Bayesian search from scikit-optimize<sup>4</sup> for full tuning, and for model tuning grid search from scikit-learn. Table 4 summarizes the tuning search space for different ML models. To mitigate excessive runtime, we chose not to tune certain ML models and limited the dataset selection to the smallest 30 for full tuning, as Table 3 illustrates.
Table 3: Factors for different tuning strategies.
Models	# Datasets
No tuning	DT, k-NN, LogReg, SVM, LGBM	50
Model tuning	DT, k-NN, LogRreg	50
Full tuning	DT, k-NN, LogReg, SVM	30
Table 4: Tuning search space.
Hyperparameter	Interval	Grid
DT	max_depth	[2, ..., 5]	{2,5, None}
k-NN	n_neighbors	[2, ..., 10]	{2,5, 10}
LogReg	C	[0.2, 5]	{0,1, 10}
SVM	C	[0.1, 2]
gamma	[0.1, 100]
4.5 Aggregating into a consensus ranking
A common practice for summarizing and interpreting the results of benchmark experiments is to aggregate them into a consensus ranking of alternatives (encoders in our case) [10, 27, 15]. To obtain a dataset-independent ranking of encoders, we aggregate the results across different datasets while keeping all other factors fixed. We now present well-known aggregation strategies used in benchmarks.
Heuristics rank alternatives based on an aggregate score. Common aggregation heuristics include mean rank (R-M) [5], median rank (R-Md), mean quality (Q-M), median quality (Q-Md), rescaled
4https://scikit-optimize.github.io/stable/
mean quality [36, 15] (Q-RM), the number of times the alternative was ranked the best (R-B) or the worst (R-W) [41], the number of times the alternative's quality is better than the best quality multiplied by a threshold  $\theta \leq 1$  ( $Q - Th_{\theta}$ ).
Friedman-Nemenyi tests [10] (R-Nem $_{p-value}$ ). First, one ranks alternatives separately for each dataset and then applies a Friedman test to reject the hypothesis that all encoders have the same average rank. If the hypothesis is rejected, pairwise Nemenyi post-hoc tests are conducted to compare pairs of alternatives. Finally, one uses the results of these post-hoc tests to construct the consensus ranking. This aggregation strategy requires the user to choose a p-value.
Kemeny-Young aggregation [21, 47] (R-Kem) first ranks alternatives separately for each dataset. Then, it determines the consensus ranking that minimizes the sum of distances to the datasets' rankings. We adopt the approach described in [45], with a distance measure that accommodates ties and missing values in the rankings. We then formulate the optimization problem as a mixed integer linear problem and solve it using a GUROBI solver with academic license. Kemeny-Young aggregation is much slower than the other aggregation strategies, taking minutes for each aggregation.
5 Results
This section summarizes the main results of our study. Appendix 7.3 further discusses the missing evaluations, run time, replicability, the ranks of the encoders and studies the effect of tuning on pipeline quality.
5.1 Sensitivity analysis
The relative performance of encoders, i.e., the ranking, can depend on the pick of ML model, quality metric, and tuning strategy. More, the choice of an aggregation strategy impacts the consensus ranking. To quantify the influence of these choices, we calculate the similarity between rankings using the Jaccard index  $J$  for the sets of best encoders and the Spearman correlation coefficient  $\rho$ . Intuitively,  $J$  measures if two experiments with different factor combinations agree on the best encoders, while  $\rho$  takes the entire ranking into account. For both measures, values close to 1 indicate high agreement and low sensitivity. Conversely, values near 0 (or, for  $\rho$ , negative) suggest low consistency and high sensitivity.
5.1.1 Sensitivity to experimental factors
We evaluate the sensitivity of encoder rankings on individual datasets with respect to an experimental factor (ML model, quality metric, or tuning strategy) by varying the factor of interest and keeping the other factors fixed, then calculating the similarity between pairs of rankings. After that, we average the result across all combinations of the other factors. Figures 1a, 1b, and 1c show the resulting values, with Spearman's  $\rho$  in the upper triangle and Jaccard index  $J$  in the lower triangle. For example, Spearman's  $\rho$  between encoder rankings for DT and SVM, averaged across all datasets, tuning strategies, and quality metrics, is 0.3.
Our findings highlight the high sensitivity of results to experimental factors, for both the full rankings and the best encoders. They also explain why results from other studies are so inconsistent, as choosing different values for any factor will lead to different results.
5.1.2 Sensitivity to aggregation strategy
To evaluate the impact of the aggregation strategy on the consensus ranking, we apply the same procedure as above to consensus rankings instead of rankings on individual datasets. Figure 1a presents the results with the notation from Section 4.5. For example, Spearman's  $\rho$  between consensus rankings obtained with Q-M and Q-Md averaged across all ML models, tuning strategies, and quality metrics is 0.8.
While some aggregation strategies show strong similarities, different strategies yield very different consensus rankings in general. This is particularly evident for Jaccard index  $J$ , indicating the high sensitivity of the best encoders to the rank aggregation strategy.
<sup>5</sup>https://www.gurobi.com/solutions/gurobi-optimizer/
Figure 1: Sensitivity as the average similarity between rankings, measured with  $\rho$  (upper triangle) and  $J$  (lower triangle), computed between individual rankings for varying: (a) ML model, (b) quality metric, (c) tuning strategy, and between consensus rankings for varying (d) aggregation strategy.
5.2 Replicability
Replicability is defined as the property of a benchmark to produce consistent results from different data [26]. This definition does not, however, provide a quantifiable notion of replicability. To overcome this, we made the following modeling decisions. First, we fix a factor combination: ML model, quality metric, tuning strategy, and aggregation strategy. We excluded the R-Nem and R-Kem aggregation strategies due to their slower run time. Second, we model the result of a benchmark on a dataset sample  $S$  with the consensus ranking aggregated across  $S$ . Third, we quantify replicability as the similarity between consensus rankings averaged over all factor combinations and 100 pairs of equal-sized disjoint sets of datasets. As discussed in Section 5.1, we measure the similarity with  $\rho$  and  $J$  to capture the similarity between both the rankings and the best encoders. We refer to them as  $\rho$ -replicability and  $J$ -replicability, respectively.
Figure 2 shows the outcome for different tuning strategies, conditional on the ML model and the size of the dataset samples. We have studied additional factors in Appendix 7.3.3. The shaded areas represent a bootstrapped  $95\%$  confidence interval. Our findings show an upward trend of  $\rho$ -replicability as the size of the dataset samples increases. This observation confirms that, in general, considering a larger number of datasets yields more reliable experimental outcomes. It is, however, important to note that this pattern does not always hold for  $J$ -replicability. This suggests that, for some models, the best encoders might vary significantly even with a relatively large number of datasets. To conclude, the replicability of our results strongly depends on the ML model, with logistic regression exhibiting the highest replicability and decision trees the lowest.
Figure 2: Replicability as the average similarity of consensus rankings from disjoint subsets of datasets.
5.3 Comparing encoders
Based on the outcome of Section 5.2, we now examine the ranks of encoders limited to decision trees, logistic regression, and all ML models.
Figure 3a shows the rank of encoders from the experiments with decision trees across all datasets, quality metrics, and tuning strategies. One-Hot is the best-performing encoder; however, Nemenyi tests at a significance level of 0.05 fail to reject that the average rank of One-Hot is the same as that of the other encoders.