Original: Feature Encodings for Gradient Boosting with Automunge
Revised: Feature Encodings for Gradient Boosting with Automunge

Original: Abstract
Revised: Abstract

Revised: Automunge is a tabular preprocessing library that encodes dataframes for supervised learning.

Original: Selecting a default feature encoding strategy for gradient boosted learning mayconsider metrics of training duration and achieved predictive performance associated with the feature representations.
Revised: When selecting a default feature encoding strategy for gradient boosted learning, one may consider metrics of training duration and achieved predictive performance associated with the feature representations.

Original: The Automungelibrary for dataframepreprocessing offers a default of binarization for categoric features and z-scorenormalization for numeric.
Revised: Automunge offers a default of binarization for categoric features and z-score normalization for numeric.

Original: The presented study sought to validate those defaultsby way of benchmarking on a series of diverse data sets by encoding variationswith tuned gradient boosted learning.
Revised: The presented study sought to validate those defaults by way of benchmarking on a series of diverse data sets by encoding variations with tuned gradient boosted learning.

Original: We found that on average our chosen defaults were top performers both from a tuning duration and a model performancestandpoint.
Revised: We found that on average our chosen defaults were top performers both from a tuning duration and a model performance standpoint.

Original: Another key finding was that one hot encoding did not perform in amanner consistent with suitability to serve as a categoric default in comparison tocategoric binarization.
Revised: Another key finding was that one hot encoding did not perform in a manner consistent with suitability to serve as a categoric default in comparison to categoric binarization.

Original: We present here these and further benchmarks
Revised: We present here these and further benchmarks.

Original:  Introduction
Revised:  Introduction

Original: The usefulness of feature engineering for applications of deep learning has long been considered asettled question in the negative, as neural networks are on their own universal function approximators Goodfellow et al. (2016).
Revised: The usefulness of feature engineering for applications of deep learning has long been considered a settled question in the negative, as neural networks are on their own universal function approximators (Goodfellow et al., 2016).

Original: However, even in the context of deep learning, tabular features are oftentreated with some form of encoding for preprocessing.
Revised: However, even in the context of deep learning, tabular features are often treated with some form of encoding for preprocessing.

Original: Automunge Authors (2021b) is a platform forencoding dataframes developed by the authors.
Revised: Automunge (Teague, 2021b) is a platform for encoding dataframes developed by the authors.

Original: This python library was originally built for a simpleuse case of basic encoding conventions for numeric and categoric features, like z-score normalizationand one-hot encodings.
Revised: This python library was originally built for a simple use case of basic encoding conventions for numeric and categoric features, like z-score normalization and one-hot encodings.

Original: Along the iterative development journey we began to flesh out a full libraryof encoding options, including a series of options for numeric and categoric features that now includescenarios for normalization, binarization, hashing, and missing data infill under automation.
Revised: Along the iterative development journey we began to flesh out a full library of encoding options, including a series of options for numeric and categoric features that now include scenarios for normalization, binarization, hashing and missing data infill under automation.

Original: Althoughit was expected that these range of encoding options would be superfluous for deep learning, thatdoes not rule out their utility in other paradigms which could range from simple regression, supportvector machines, decisions trees, or as will be the focus of this paper, gradient boosting.
Revised: Although it was expected that these range of encoding options would be superfluous for deep learning, that does not rule out their utility in other paradigms which could range from simple regression, support vector machines, decisions trees or as will be the focus of this paper, gradient boosting.

Original: The purpose of this work is to present the results of a benchmarking study between alternateencoding strategies for numeric and categoric features for gradient boosted tabular learning.
Revised: The purpose of this work is to present the results of a benchmarking study between alternate encoding strategies for numeric and categoric features for gradient boosted tabular learning.

Original: We wereparticularly interested in validating the library’s default encoding strategies, and found that in bothprimary performance metrics of tuning duration time and model performance the current defaultsunder automation of categoric binarization and numeric z-score normalization demonstrated merit toserve as default encodings for the Automunge library.
Revised: We were particularly interested in validating the library’s default encoding strategies, and found that in both primary performance metrics of tuning duration time and model performance the current defaults under automation of categoric binarization and numeric z-score normalization demonstrated merit to serve as default encodings for the Automunge library.

Original: We also found that in addition to our defaultbinarization, even a frequency sorted variant of ordinal encoding on average outperformed one hotencoding.
Revised: We also found that in addition to our default binarization, even a frequency sorted variant of ordinal encoding on average outperformed one hot encoding. Has  Automunge

Revised: Automunge (Teague, 2021b) is an open source python library, available now for pip install, built on top of Pandas (McKinney, 2010), Numpy (Harris et al., 2020), SciKit-learn (Pedregosa et al., 2011), and Scipy (Virtanen et al., 2020).

Revised: It takes as input tabular data received in a tidy form, meaning one column per feature and one row per sample, and returns numerically encoded sets with infill to missing points, thus providing a push-button means to feed raw tabular data directly to machine learning.

Original: Gradient Boosting
Revised: The extent of derivations may be minimal, such as numeric normalizations and categoric binarizations under automation, or may include more elaborate univariate transformations, including aggregated sets thereof. Generally speaking, the transformations are performed based on a fit to properties of features in a designated training set, and then that same basis may be used to consistently and efficiently prepare subsequent test data, as may be intended for use in inference or for additional training data preparation. The interface is channeled through two master functions, automunge(.) and postmunge(.). The automunge(.) function receives a training set and if available also a consistently formatted test set, and returns a collection of dataframes intended for training, validation, and inference — each of these aggregations further segregated into subsets of features, index, and label sets.

Revised: A validation set, if designated by ratio of partitioned data from the training set, is segregated from the training data prior to transformations and then consistently prepared on the train set basis to avoid data leakage between training and validation.

Revised: The function also returns a populated python dictionary, which we call the postprocess_dict, recording steps and parameters of transformations.

Revised: This dictionary may then be passed along with subsequent test data to the postmunge(.) function for consistent preparations on the train set basis, as for instance may be applied sequentially to streams of data.

Revised: Because it makes use of train set properties evaluated during a corresponding automunge(.) call instead of directly evaluating properties of the test data, preparing data in the postmunge(.) function can be very efficient.

Revised: There is a built in extensive library of feature encodings to choose from.

Revised: Numeric features may be assigned to any range of transformations, normalizations, and bin aggregations.

Revised: Sequential numeric features may be supplemented by proxies for derivatives (Teague, 2022b).

Revised: Categoric features may be encoded as ordinal, one hot, binarization, hashing, or even parsed categoric encoding (Teague, 2022c) with an increased information retention in comparison to one hot encoding by a vectorization as a function of grammatical structure shared between entries.

Revised: Categoric sets may be collectively aggregated into a single common binarization.

Original: Gradient boosting Friedman (2000) refers to a paradigm of decision tree learning Quinlan (1986)similar to random forests Breiman (2001) but in which the optimization is boosted by recursivelytraining an iteration’s model objective to correct the performance of  the preceding iteration’s model.
Revised: Categoric labels may have label smoothing applied (Szegedy et al., 2016), or fitted smoothing where null values are fit to class distributions. Sets of transformations to be directed at targeted features can be assembled which include generations and branches of derivations by making use of our family tree primitives (Teague, 2021b), as can be used to redundantly encode a feature in multiple configurations of varying information content. Such transformation sets may be accessed from those predefined in an internal library for simple assignment or alternatively may be custom configured. Even the transformation functions themselves may be custom defined from a very simple template.

Original: It is commonly implemented in practice by the XGBoost library Chen & Guestrin (2016) for GPUacceleration, although there are architecture variations available for different fortes, like LightGPM
Revised: Through application statistics of the features are recorded to facilitate detection of distribution drift. Inversion is available to recover the original form of data found preceding transformations, as may be used to recover the original form of labels after inference. Missing data is imputed by auto ML models trained on surrounding features (Teague, 2022a).

Original: Ke et al.
Revised: Noise may be channeled into feature encodings for non-deterministic inference, as may include stochastic perturbations sampled from quantum circuits (Teague, 2022d).  Gradient Boosting Gradient boosting (Friedman, 2000) refers to a paradigm of decision tree learning (Quinlan, 1986) similar to random forests (Breiman, 2001) but in which the optimization is boosted by recursively training an iteration’s model objective to correct the performance of the preceding iteration’s model.

Original: (2017) which may train faster on CPU’s than XGBoost (with a possible performancetradeoff).
Revised: It is commonly implemented in practice by the XGBoost library (Chen & Guestrin, 2016) for GPU acceleration, although there are architecture variations available for different fortes, like LightGPM (Ke et al., 2017) which may train faster on CPU’s than XGBoost (with a possible performance tradeoff).

Original: Gradient boosting has traditionally been found as a winning solution for tabular modality competitions on the Kaggle platform, and its competitive efficacy has even been demonstrated for moresophisticated applications like time series sequential learning when used for window based regression
Revised: Gradient boosting has traditionally been found as a winning solution for tabular modality competitions on the Kaggle platform, and its competitive efficacy has even been demonstrated for more sophisticated applications like time series sequential learning when used for window based regression

Original: Elsayed et al.
Revised: (Elsayed et al., 2021).

Original: Recent tabular benchmarking papers have found that gradient boosting maystill mostly outperform sophisticated neural architectures like transformers Gorishniy et al. (2021),although even a vanilla multi layer perceptron neural network could have capacity to outperformgradient boosting with comprehensively tuned regularizers Kadra et al.
Revised: Recent tabular benchmarking papers have found that gradient boosting may still mostly outperform sophisticated neural architectures like transformers (Gorishniy et al., 2021), although even a vanilla multi layer perceptron neural network could have capacity to outperform gradient boosting with comprehensively tuned regularizers (Kadra et al., 2021).

Original: Gradient boostingcan also be expected to have higher latency inference than neural networks Borisov et al.
Revised: Gradient boosting can also be expected to have higher latency inference than neural networks (Borisov et al., 2021).

Original: Conventional wisdom is that one can expect gradient boosting models to have capacity for betterperformance than random forests for tabular applications but with a tradeoff of increased probabilityof overfitting without hyperparameter tuning Howard & Gugger (2020).
Revised: Conventional wisdom is that one can expect gradient boosting models to have capacity for better performance than random forests for tabular applications but with a tradeoff of increased probability of overfitting without hyperparameter tuning (Howard & Gugger, 2020).

Original: With both more sensitivityto tuning parameters and a much higher number of parameters in play than random forest, gradientboosting usually requires more sophistication than a simple grid or random search for tuning.
Revised: With both more sensitivity to tuning parameters and a much higher number of parameters in play than random forest, gradient boosting usually requires more sophistication than a simple grid or random search for tuning.

Original: Onecompromise method available is for a sequential grid search through different subsets of parameters Jain (2016), although more automated and even parallelized methods are available by way of blackbox optimization libraries like Optuna Akiba et al.
Revised: One compromise method available is for a sequential grid search through different subsets of parameters (Jain, 2016), although more automated and even parallelized methods are available by way of black box optimization libraries like Optuna (Akiba et al., 2019).

Original: There will likely be more improvementsto come both in libraries and tuning conventions, this is an active channel of industry research.
Revised: There will likely be more improvements to come both in libraries and tuning conventions, this is an active channel of industry research.

Original:  Feature Encodings
Revised:  Feature Encodings

Original: Feature encoding refers to feature set transformations that serve to prepare the data for machinelearning.
Revised: Feature encoding refers to feature set transformations that serve to prepare the data for machine learning.

Original: Common forms of feature encoding preparations include normalizations for numeric setsand one hot encodings for categoric, although some learning libraries may accept categoric featuresin string representations for internal encodings.
Revised: Common forms of feature encoding preparations include normalizations for numeric sets and one hot encodings for categoric, although some learning libraries may accept categoric features in string representations for internal encodings.

Original: Before the advent of deep learning, it was commonto supplement features with alternate representations of extracted information or to combine featuresin some fashion.
Revised: Before the advent of deep learning, it was common to supplement features with alternate representations of extracted information or to combine features in some fashion.

Original: Such practices of feature engineering are sometimes still applied in gradient boostedlearning, and it was one of the purposes of these benchmarks to evaluate benefits of the practice incomparison to directly training on the data.
Revised: Such practices of feature engineering are sometimes still applied in gradient boosted learning, and it was one of the purposes of these benchmarks to evaluate benefits of the practice in comparison to directly training on the data.

Original: An important distinction of feature encodings can be considered as those that can be applied independent of an esoteric domain profile verses those that rely on external structure.
Revised: An important distinction of feature encodings can be considered as those that can be applied independent of an esoteric domain profile verses those that rely on external structure.

Original: An example couldbe the difference between supplementing a feature with bins derived based on the distribution ofpopulated numeric values verses extracting bins based on an external database lookup.
Revised: An example could be the difference between supplementing a feature with bins derived based on the distribution of populated numeric values verses extracting bins based on an external database lookup.

Original: In the caseof Automunge, the internal library of encodings follows almost exclusively the former, that is mostencodings are based on inherent numeric or string properties and do not consider adjacent propertiesthat could be inferred based on relevant application domains.
Revised: In the case of Automunge, the internal library of encodings follows almost exclusively the former, that is most encodings are based on inherent numeric or string properties and do not consider adjacent properties that could be inferred based on relevant application domains.

Original: (An exception is made for date-timeformatted features which under automation automatically extract bins for weekdays, business hours,holidays, and redundantly encodes entries based on cyclic periods of different time scales London(2016).)
Revised: (An exception is made for date-time formatted features which under automation automatically extract bins for weekdays, business hours, holidays, and redundantly encodes entries based on cyclic periods of different time scales (London, 2016).)

Original: The library includes a simple template for integrating custom univariate transformations Authors (2021a) if a user would like to integrate into a pipeline alternate conventions.
Revised: The library includes a simple template for integrating custom univariate transformations (Teague, 2021a) if a user would like to integrate into a pipeline alternate conventions.

Original: 3.1 Numeric
Revised: 4.1 Numeric

Original: Numeric normalizations  in practice are most commonly applied similar to our default of z-score ‘nmbr’(subtract mean and divide by standard deviation) or min-max scaling ‘mnmx’ (converting to rangebetween 0–1).
Revised: Numeric normalizations [Appendix A] in practice are most commonly applied similar to our default of z-score ‘nmbr’ (subtract mean and divide by standard deviation) or min-max scaling ‘mnmx’ (converting to range between 0–1).

Original: Other variations that may be found in practice include mean scaling ‘mean’(subtractmean and divide by min max delta), and max scaling ‘mxab’ (divide by feature set absolute max).
Revised: Other variations that may be found in practice include mean scaling ‘mean’(subtract mean and divide by min max delta), and max scaling ‘mxab’ (divide by feature set absolute max).

Original: More sophisticated conventions may convert a distribution shape in addition to the scale, such asthe box-cox power law transformation ‘bxcx’ Box & Cox
Revised: More sophisticated conventions may convert a distribution shape in addition to the scale, such as the box-cox power law transformation ‘bxcx’ (Box & Cox, 1964) or Scikit-Learn’s

Original: (2011) quantile transformer ‘qttf’, which both may serve the purpose of converting a feature set tocloser resemble a Gaussian distribution.
Revised: (Pedregosa et al., 2011) quantile transformer ‘qttf’, which both may serve the purpose of converting a feature set to closer resemble a Gaussian distribution.

Original: In general, numeric normalizations are more commonlyapplied for learning paradigms other than those based on decision trees, where for example in neuralnetworks they serve the purpose of normalizing gradient updates across features.
Revised: In general, numeric normalizations are more commonly applied for learning paradigms other than those based on decision trees, where for example in neural networks they serve the purpose of normalizing gradient updates across features.

Original: We did find thatthe type of normalizations applied to numeric features appeared to impact performance, and we willpresent these findings below.
Revised: We did find that the type of normalizations applied to numeric features appeared to impact performance, and we will present these findings below.

Original: 3.2 Categoric
Revised: 4.2 Categoric

Original: Categoric encodings  are most commonly derived in practice as a one hot encoding, where each uniqueentry in a received feature is translated to boolean integer activations in a dedicated column among areturned set thereof.
Revised: Categoric encodings [Appendix B] are most commonly derived in practice as a one hot encoding, where each unique entry in a received feature is translated to boolean integer activations in a dedicated column among a returned set thereof.

Original: The practice of one hot encoding has shortcomings in the high cardinality case(where a categoric feature has an excessive number of unique entries), which in the context of gradientboosting may be particularly impactful as an inflated column count impairs latency performanceof a training operation — or when the feature is targeted as a classification label may even causetraining to exceed memory overhead constraints.
Revised: The practice of one hot encoding has shortcomings in the high cardinality case (where a categoric feature has an excessive number of unique entries), which in the context of gradient boosting may be particularly impactful as an inflated column count impairs latency performance of a training operation — or when the feature is targeted as a classification label may even cause training to exceed memory overhead constraints.

Original: The Automunge library attempts to circumvent thishigh cardinality edge case in two fashions, first by defaulting to a binarization encoding instead ofone hot, and second by distinguishing highest cardinality sets for a hashed encoding Moody (1988)
Revised: The Automunge library attempts to circumvent this high cardinality edge case in two fashions, first by defaulting to a binarization encoding instead of one hot, and second by distinguishing highest cardinality sets for a hashed encoding (Moody, 1988)

Original: Weinberger et al. (2009) Authors (2021a) which may stochastically consolidate multiple uniqueentries into a shared ordinal representation for a reduced number of unique entries.
Revised: (Weinberger et al., 2009) (Teague, 2020) which may stochastically consolidate multiple unique entries into a shared ordinal representation for a reduced number of unique entries.

Original: The library default of categoric binarization ‘1010’ refers to translating each unique entry in areceived feature to a unique set of zero, one, or more boolean integer activations in a returned setof boolean integer columns.
Revised: The library default of categoric binarization ‘1010’ refers to translating each unique entry in a received feature to a unique set of zero, one, or more boolean integer activations in a returned set of boolean integer columns.

Original: Where one hot encoding may return a set of n columns for n uniqueentries, binarization will instead return a smaller count of log2(n) rounded up to nearest integer.
Revised: Where one hot encoding may return a set of n columns for n unique entries, binarization will instead return a smaller count of log2(n) rounded up to nearest integer.

Original: Wehave previously seen the practice discussed in the blogging literature, such as Ravi (2019), althoughwithout validation as offered herein.
Revised: We have previously seen the practice discussed in the blogging literature, such as (Ravi, 2019), although without validation as offered herein.

Original: A third common variation on categoric representations includes ordinal encodings, which simplyrefers to returning a single column encoding of a feature with a distinct integer representation foreach unique entry.
Revised: A third common variation on categoric representations includes ordinal encodings, which simply refers to returning a single column encoding of a feature with a distinct integer representation for each unique entry.

Original: Variations on ordinal encodings in the library may sort the integer representationsby frequency of the unique entry ‘ord3’ or based on alphabetic sorting ‘ordl’.
Revised: Variations on ordinal encodings in the library may sort the integer representations by frequency of the unique entry ‘ord3’ or based on alphabetic sorting ‘ordl’.

Original: Another convention for categoric sets unique to the Automunge library we refer to as parsed categoricencodings ‘or19’ Authors (2020).
Revised: Another convention for categoric sets unique to the Automunge library we refer to as parsed categoric encodings ‘or19’ (Teague, 2022c).

Original: Parsed encodings search through tiers of string character subsets ofunique entries to identify shared grammatical structure for supplementing encodings with structurederived from a training set basis.
Revised: Parsed encodings search through tiers of string character subsets of unique entries to identify shared grammatical structure for supplementing encodings with structure derived from a training set basis.

Original: Parsed encodings are supplemented with extracted numeric portionsof unique entries for additional information retention in the form received by training.
Revised: Parsed encodings are supplemented with extracted numeric portions of unique entries for additional information retention in the form received by training.

Original:  Benchmarking
Revised:  Benchmarking

Original: The benchmarking sought to evaluate a range of numeric and categoric encoding scenarios by way oftwo key performance metrics, training time and model performance.
Revised: The benchmarking sought to evaluate a range of numeric and categoric encoding scenarios by way of two key performance metrics, training time and model performance.

Original: Training was performed overthe course of 1.5 weeks on a Lambda workstation with AMD 3970X processor, 128Gb RAM, andtwo Nvidia 3080 GPUs.
Revised: Training was performed over the course of 1.5 weeks on a Lambda workstation with AMD 3970X processor, 128Gb RAM, and two Nvidia 3080 GPUs.

Original: Training was performed by way of XGBoost tuned by Optuna with 5-foldfast cross-validation Swersky et al. and early stopping criteria of 50 tuning iterations withoutimprovement.
Revised: Training was performed by way of XGBoost tuned by Optuna with 5-fold fast cross-validation (Swersky et al., 2013) and early stopping criteria of 50 tuning iterations without improvement.

Original: Performance was evaluated against a partitioned 25% validation set based on a f1score performance metric, which we understand is a good default for balanced evaluation of biasand variance performance of classification tasks Stevens et al.
Revised: Performance was evaluated against a partitioned 25% validation set based on a f score performance metric, which we understand is a good default for balanced evaluation of bias and variance performance of classification tasks (Stevens et al., 2020).

Original: This loop was repeated andaveraged across 5 iterations and then repeated and averaged across 31 tabular classification data setssourced from the OpenML benchmarking repository Vanschoren et al.
Revised: This loop was repeated and averaged across 5 iterations and then repeated and averaged across 31 tabular classification data sets sourced from the OpenML benchmarking repository (Vanschoren et al., 2014).

Original: Rephrasing for clarity,the reported metrics are averages of 5 repetitions of 31 data sets for each encoding type as applied toall numeric or categoric features for training.
Revised: Rephrasing for clarity, the reported metrics are averages of 5 repetitions of 31 data sets for each encoding type as applied to all numeric or categoric features for training.

Original: The distribution bands shown in the figures are acrossthe five repetitions.
Revised: The distribution bands shown in the figures are across the five repetitions.

Original: The data sets were selected for diverse tabular classification applications within-memory scale training data and tractable label cardinality.
Revised: The data sets were selected for diverse tabular classification applications with in-memory scale training data and tractable label cardinality.

Original: We found that these benchmarks gave us comfort in the Automunge library’s defaults of numericz-score normalization and categoric binarization.
Revised: We found that these benchmarks gave us comfort in the Automunge library’s defaults of numeric z-score normalization and categoric binarization.

Original: An interesting result was the outperformanceof categoric binarization in comparison to one-hot encoding, as the latter is commonly used in[Figure] [Figure] mainstream practice as a default.
Revised: An interesting result was the outperformance of categoric binarization in comparison to one-hot encoding, as the latter is commonly used in mainstream practice as a default.

Original: Further dialogue for the interpretation of the results presented in
Revised: Further dialogue for the interpretation of the results presented in

Original: Figures 1 and 2 are provided in the Appendix.
Revised: Figures 1 and 2 are provided in [Appendix A, B].

Original:  Conclusion
Revised:  Conclusion

Original: We hope that these benchmarks may have provided some level of user comfort by validating thedefault encodings applied under automation by the Automunge library of z-score normalization andcategoric binarization, both from a training time and model performance standpoint.
Revised: We hope that these benchmarks may have provided some level of user comfort by validating the default encodings applied under automation by the Automunge library of z-score normalization and categoric binarization, both from a training time and model performance standpoint.

Original: If you wouldlike to try out the library we recommend the tutorials folder found on GitHub Authors (2021b) as astarting point. 
Revised: If you would like to try out the library we recommend the tutorials folder found on GitHub (Teague, 2021b) as a starting point. [Figure] [Figure]

Original: A Numeric Results
Revised: A Numeric Encodings A.1 default

Original: A.1 default

Original: • defaults for Automunge under automation as z-score normalization (‘nmbr’ code in the library) • The default encoding was validated both from a tuning duration and a model performancestandpoint as top performing scenario on average.
Revised: • defaults for Automunge under automation as z-score normalization (‘nmbr’ code in the library) • The default encoding was validated both from a tuning duration and a model performance standpoint as top performing scenario on average.

Original: A.2 qttf
Revised: A.2 qttf

Original: • Scikit-Learn QuantileTransformer with a normal output distribution • The quantile distribution conversion did not perform as well on average as simple z-scorenormalization, although it remained a top performer.
Revised: • Scikit-Learn QuantileTransformer with a normal output distribution • The quantile distribution conversion did not perform as well on average as simple z-score normalization, although it remained a top performer.

Original: A. powertransform
Revised: A.3 powertransform

Original: • the Automunge option to conditionally encode between ‘bxcx’, ‘mmmx’, or ‘MAD3’ basedon distribution properties (via library’s powertransform=
Revised: • the Automunge option to conditionally encode between ‘bxcx’, ‘mmmx’, or ‘MAD3’ based on distribution properties (via library’s powertransform=

Original: True setting)
Revised: True setting)

Original: • This was the worst performing encoding scenario, which at a minimum demonstrates thatthe heuristics and statistical measures currently applied by the library to conditionally selecttypes of encodings could use some refinement.
Revised: • This was the worst performing encoding scenario, which at a minimum demonstrates that the heuristics and statistical measures currently applied by the library to conditionally select types of encodings could use some refinement.

Original: A.4 mnmx
Revised: A.4 mnmx

Original: • min max scaling ‘mnmx’ which shifts a feature distribution into the range 0–1• This scenario performed considerably worse than z-score normalization, which we expectwas due to cases where outlier values may have caused the predominantly populated regionto get “squished together” in the encoding space.
Revised: • min max scaling ‘mnmx’ which shifts a feature distribution into the range 0– • This scenario performed considerably worse than z-score normalization, which we expect was due to cases where outlier values may have caused the predominantly populated region to get “squished together” in the encoding space.

Original: A.5 capped quantiles
Revised: A.5 capped quantiles

Original: • min max scaling with capped outliers at 0.99 and 0.01 quantiles (‘mnm3’ code in library)• This scenario is best compared directly to min-max scaling, and demonstrates that defaultingto capping outliers did not benefit performance on average.
Revised: • min max scaling with capped outliers at 0.99 and 0.01 quantiles (‘mnm3’ code in library) • This scenario is best compared directly to min-max scaling, and demonstrates that defaulting to capping outliers did not benefit performance on average.

Original: A.6 binstransform• z-score normalization supplemented by 5 one hot encoded standard deviation bins (vialibrary’s binstransform=
Revised: • z-score normalization supplemented by 5 one hot encoded standard deviation bins (via library’s binstransform=

Original: True setting)•
Revised: True setting)

Original: In addition to a widened range of tuning durations, the supplemental bins did not appear tobe beneficial to model performance for gradient boosting.
Revised: In addition to a widened range of tuning durations, the supplemental bins did not appear to be beneficial to model performance for gradient boosting.

Original: B Categoric Results
Revised: B Categoric Encodings B.1 default

Original: B.1 default

Original: • defaults for Automunge under automation for categoric binarization (‘1010’ code in thelibrary) • The default encoding was validated as top performing both from a tuning duration and amodel performance standpoint.
Revised: • defaults for Automunge under automation for categoric binarization (‘1010’ code in the library) • The default encoding was validated as top performing both from a tuning duration and a model performance standpoint.

Original: B.2 onht
Revised: B.2 onht

Original: • one hot encoding • The model performance impact was surprisingly negative compared to the default considering this is often used as a default in mainstream practice.
Revised: • one hot encoding • The model performance impact was surprisingly negative compared to the default considering this is often used as a default in mainstream practice.

Original: Based on this benchmark werecommend discontinuing use of one-hot encoding outside of special use cases (like e.g. forpurposes of feature importance analysis).
Revised: Based on this benchmark we recommend discontinuing use of one-hot encoding outside of special use cases (like e.g. for purposes of feature importance analysis).

Original: B.3 [-ord
-] 
Revised: B.3 [-ord
-]ord

Original: • ordinal encoding with integers sorted by category frequency ‘ord3’ • Sorting ordinal integers by category frequency instead of alphabetic significantly benefitedmodel performance, in most cases lifting ordinal above one hot encoding although still notin the range of the default binarization.
Revised: • ordinal encoding with integers sorted by category frequency ‘ord3’ • Sorting ordinal integers by category frequency instead of alphabetic significantly benefited model performance, in most cases lifting ordinal above one hot encoding although still not in the range of the default binarization.

Original: B.4 ordl
Revised: B.4 ordl

Original: • ordinal encoding with integers sorted alphabetically by category ‘ordl’ • Alphabetic sorted ordinal encodings (as is the default for Scikit-Learn’s OrdinalEncoder) didnot perform as well, we recommend defaulting to frequency sorted integers when applyingordinal.
Revised: • ordinal encoding with integers sorted alphabetically by category ‘ordl’ • Alphabetic sorted ordinal encodings (as is the default for Scikit-Learn’s OrdinalEncoder) did not perform as well, we recommend defaulting to frequency sorted integers when applying ordinal.

Original: B.5 hsh 
Revised: B.5 hsh

Original: • hashed ordinal encoding (library default for high cardinality categoric ‘hsh2’)
Revised: • hashed ordinal encoding (library default for high cardinality categoric ‘hsh2’)

Original: • This benchmark was primarily included for reference, it was expected that as some categories may be consolidated there would be a performance impact for low cardinality sets.
Revised: • This benchmark was primarily included for reference, it was expected that as some categories may be consolidated there would be a performance impact for low cardinality sets.

Original: Thebenefit of hashing is for high cardinality which may otherwise impact gradient boosting memory overhead.
Revised: The benefit of hashing is for high cardinality which may otherwise impact gradient boosting memory overhead.

Original: B.6 or 
Revised: B.6 or

Original: • multi-tier string parsing ‘or19’ Authors (2020)• It appears that our recent invention of multi-tier string parsing succeeded in outperforming
Revised: • multi-tier string parsing ‘or19’ (Teague, 2022c) •

Original:  one-hot encoding and was the second top performer, but did not perform sufficiently to recommended defaulting in comparison to vanilla binarization.
Revised: It appears that our recent invention of multi-tier string parsing succeeded in outperforming one-hot encoding and was the second top performer, but did not perform sufficiently to recommended defaulting in comparison to vanilla binarization.

Original: We recommend reserving string parsing for cases where the application may have some extended structure associatedwith grammatical content, as was validated as outperforming 
Revised: We recommend reserving string parsing for cases where the application may have some extended structure associated with grammatical content, as was validated as outperforming binarization for an example in the citation.

Original: C Data Sets
Revised: C Data Sets

Original: The Benchmarking included the following tabular data sets, shown here with their OpenML IDnumber.
Revised: The Benchmarking included the following tabular data sets, shown here with their OpenML ID number.

Original: A thank you to Vanschoren et al. (2014) for providing the data sets and Kadra et al. (2021)for inspiring the composition.
Revised: A thank you to (Vanschoren et al., 2014) for providing the data sets and (Kadra et al., 2021) for inspiring the composition.

Original: • Click prediction /
Revised: • Click prediction /•

Original: • C.C.FraudD. / 233143• sylvine / 233135• jasmine / 233134• fabert / 233133• APSFailure / 233130• MiniBooNE / 233126• volkert / 233124• jannis / 233123• numerai28.6 / 233120• Jungle-Chess-2pcs / 233119• segment / 233117• car / 233116• Australian / 233115• higgs / 233114• shuttle / 233113• connect-4 / 233112• bank-marketing / 233110• blood-transfusion / 233109• nomao / 233107• ldpa / 233106• skin-segmentation /
Revised: C.C.FraudD. /• sylvine /• jasmine /• fabert /• APSFailure /• MiniBooNE /• volkert /• jannis /• numerai28.6 /• Jungle-Chess-2pcs /• segment /• car /•

Revised: Australian /• higgs /• shuttle /•

Original: 233104• phoneme / 233103• walking-activity / 233102• adult / 233099• kc1 / 233096• vehicle / 233094• credit-g / 233088• mfeat-factors / 233093• arrhythmia / 233092• kr-vs-kp /[-
-]
Revised: connect-4 /• bank-marketing /• blood-transfusion /• nomao /• ldpa /• skin-segmentation /• phoneme /• walking-activity /• adult /• kc1 /• vehicle /• credit-g /• mfeat-factors /• arrhythmia /• kr-vs-kp /[-
-]

Original:  Checklist
Revised: D Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?[N/A]
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original:  (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [No] (see supplemental material notebooks)
Revised: [No] (see supplemental material notebooks)

Original: (c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 
Revised: [Yes] 4.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [No] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [No] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [No] 
Revised: [No] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]