Original: L ABEL E NCODING FOR R EGRESSION N ETWORKS
Revised: L ABEL E NCODING FOR R EGRESSION N ETWORKS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Deep neural networks are used for a wide range of regression problems and are typically trained by minimizing the squared or absolute error of output labels.
Revised: Deep neural networks are used for a wide range of regression problems. However, there exists a signiﬁcant gap in accuracy between specialized approaches and generic direct regression in which a network is trained by minimizing the squared or absolute error of output labels.

Original: Solving a regression problem with a set of binary classiﬁers can improvethe accuracy and also utilizes well-studied binary classiﬁcation algorithms.
Revised: Prior work has shown that solving a regression problem with a set of binary classiﬁers can improve accuracy by utilizing wellstudied binary classiﬁcation algorithms.

Original: However, a limited design space of regression by binary classiﬁcation has been studied by prior works.

Original: This work introduces a novel approach using binary-encoded labels (BEL) to generalize the application of binary classiﬁcation to regression, and a taxonomy identifying the key design aspects of such formulation.
Revised: We introduce binary-encoded labels (BEL) , which generalizes the application of binary classiﬁcation to regression by providing a framework for considering arbitrary multi-bit values when encoding target values.

Original: Of these design aspects, we primarily explore and identifythe desirable properties of suitable encoding and decoding functions used for the conversion between real-valued and binary-encoded labels based on theoretical and empirical study. 
Revised: We identify desirable properties of suitable encoding and decoding functions used for the conversion between real-valued and binary-encoded labels based on theoretical and empirical study. These properties highlight a tradeoff between classiﬁcation error probability and error-correction capabilities of label encodings.

Original: BEL can be combined with off-the-shelf task-speciﬁc feature extractors and trained end-toend.
Revised: BEL can be combined with off-the-shelf task-speciﬁc feature extractors and trained end-to-end.

Original: Wefurther propose a series of sample encoding, decoding, and training loss functions applicable in this domain and demonstrate that our generic regression approach results in lower error than direct regression and specialized approaches while being suitable for a diverse set of regression problems, network architectures, and evaluation metrics. 
Revised: We propose a series of sample encoding, decoding, and training loss functions for BEL and demonstrate they result in lower error than direct regression and specialized approaches while being suitable for a diverse set of regression problems, network architectures, and evaluation metrics. BEL achieves state-of-the-art accuracies for several regression benchmarks. Code is available at https://github.com/ubc-aamodt-group/BEL_regression .

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep regression networks, in which a continuous output is predicted for a given input, are traditionally trained by minimizing squared/absolute error of a logit, which we refer to as direct regression .
Revised: Deep regression networks, in which a continuous output is predicted for a given input, are traditionally trained by minimizing squared/absolute error of output labels, which we refer to as direct regression .

Original: More recently, several works have proposed specialized regression  approaches for regression problems such as head pose estimation, age estimation, and facial landmark estimation, and shown signiﬁcant accuracy improvements over direct regression.
Revised: However, there is a signiﬁcant gap in accuracy between direct regression and recent task-specialized approaches for regression problems including head pose estimation, age estimation and facial landmark estimation.

Original: However, developingtask-agnostic generic approaches to improve accuracyfor regression is desirable.
Revised: Given the increasing importance of deep regression networks, developing generic approaches to improving their accuracy is desirable.

Original: A regression problem can be posed as a set of binary classiﬁcation sub-problems.
Revised: A regression problem can be posed as a set of binary classiﬁcation problems.

Original: A similar approach has been applied to other domains such as ordinal regression (Li & Lin, 2006) and multiclass classiﬁcation (Dietterich & Bakiri, 1995).
Revised: A similar approach has been applied to other domains such as ordinal regression (Li & Lin, 2006) and multiclass classiﬁcation (Dietterich & Bakiri, 1995).

Original: Such  formulation allows the use of well-studied binary classiﬁcation approaches.
Revised: Such a formulation allows the use of well-studied binary classiﬁcation approaches.

Original: Further, new generalization bounds for ordinal regression or multiclass classiﬁcation can be derived from the known generalization bounds of binary classiﬁcation.
Revised: Further, new generalization bounds for ordinal regression or multiclass classiﬁcation can be derived from the known generalization bounds of binary classiﬁcation.

Original: This reduces the efforts for design, implementation, and theoretical analysis signiﬁcantly (Li & Lin, 2006).
Revised: This reduces the efforts for design, implementation, and theoretical analysis signiﬁcantly (Li & Lin, 2006).

Original: Dietterich & Bakiri (1995) demonstrated that posing multiclass classiﬁcation as a set of binary classiﬁcation problems can increase error tolerance and improve accuracy.
Revised: Dietterich & Bakiri (1995) demonstrated that posing multiclass classiﬁcation as a set of binary classiﬁcation problems can increase error tolerance and improve accuracy.

Original: However, the proposed approaches for multiclass classiﬁcation do not apply to regression due to the differences in task objective and properties of the  error probability distributionof classiﬁers (Section 2).
Revised: However, the proposed approaches for multiclass classiﬁcation do not apply to regression due to the differences in task objective and properties of the classiﬁers’ error probability distribution (Section 2).

Original: On the other hand, prior works on ordinal regression have explored the application of binary classiﬁers in a more restricted way which limits its application to a wide range of complex regression problems (Section 2).
Revised: On the other hand, prior works on ordinal regression have explored the application of binary classiﬁers in a more restricted way which limits its application to a wide range of complex regression problems (Section 2).

Original: There exists a lack of a generic framework that uniﬁes possible formulations for using binary classiﬁcation to solve regression.
Revised: There exists a lack of a generic framework that uniﬁes possible formulations for using binary classiﬁcation to solve regression.

Original: In this work, we propose binary-encoded labels (BEL) which improves regression by generalizingthe application of binary classiﬁcation.
Revised: In this work, we propose binary-encoded labels (BEL) which improves accuracy by generalizing application of binary classiﬁcation to regression.

Original: In BEL, a target label is quantized and converted to a binary code of length M , and M binary classiﬁers are then used to learn  binary-encoded labels.
Revised: In BEL, a target label is quantized and converted to a binary code of length M , and M binary classiﬁers are then used to learn these binary-encoded labels.

Original: An encoding function is introduced to convert the target label to a binary code, and a decoding function is introduced to decode the output of binary classiﬁers to the predicted label.
Revised: An encoding function is introduced to convert the target label to a binary code, and a decoding function is introduced to decode the output of binary classiﬁers to a real-valued prediction.

Original: BEL allows using an adjustable number of binary classiﬁers depending upon the quantization, encoding, and decoding functions.
Revised: BEL allows using an adjustable number of binary classiﬁers depending upon the quantization, encoding, and decoding functions.

Original: BEL opens possible avenues to improve the accuracy of regression problems with a large design space of the quantization function, encoding/decoding functions, and optimization algorithm.
Revised: BEL opens possible avenues to improve the accuracy of regression problems with a large design space spanning quantization, encoding, decoding, and loss functions.

Original: We focus on the encoding and decoding functions and theoretically study the relations between the absolute error of label and binary classiﬁers’ errors for sample encoding and decoding functions.
Revised: We focus on the encoding and decoding functions and theoretically study the relations between the absolute error of label and binary classiﬁers’ errors for sample encoding and decoding functions.

Original: This analysis demonstrates the impact of binary classiﬁers’ error distribution over the numeric range of target labels on the suitability of different encoding and decoding functions.
Revised: This analysis demonstrates the impact of binary classiﬁers’ error distribution over the numeric range of target labels on the suitability of different encoding and decoding functions.

Original: Based on our analysis and empirically observed binary classiﬁers’ error distribution, we propose properties of suitable encoding functions for regression and explore various encoding functions on a wide range of tasks.
Revised: Based on our analysis and empirically observed binary classiﬁers’ error distribution, we propose properties of suitable encoding functions for regression and explore various encoding functions on a wide range of tasks.

Original: We also propose an expected correlation-based decoding function for regression that can effectively reduce the quantization error introduced by the encoding.
Revised: We also propose an expected correlation-based decoding function for regression that can effectively reduce the quantization error introduced by the use of classiﬁcation.

Original: A deep regression network consists of a feature extractor and a regressor and is trained end-toend.
Revised: A deep regression network consists of a feature extractor and a regressor and is trained end-to-end.

Revised: A regressor is typically the last fully connected layer with one output logit for direct regression.

Original: Our proposed regression approach (BEL) can be combined with off-the-shelf task-speciﬁc feature extractors by increasing the regressor’s output logits.
Revised: Our proposed regression approach (BEL) can be combined with off-the-shelf task-speciﬁc feature extractors by increasing the regressor’s output logits.

Original: We ﬁnd that the correlation between multiple binary classiﬁers’ outputs can be exploited to reduce the size of the feature vector and consequently reduce the number of parameters in the regressor.
Revised: Further, we ﬁnd that the correlation between multiple binary classiﬁers’ outputs can be exploited to reduce the size of the feature vector and consequently reduce the number of parameters in the regressor.

Original: We explore the use of  decoding functions for training loss formulation and evaluatethe use of the binary cross-entropy, cross-entropy, and squared/absolute error loss functions for BEL.
Revised: We explore the use of different decoding functions for training loss formulation and evaluate binary cross-entropy, cross-entropy, and squared/absolute error loss functions for BEL.

Original: We evaluate our approach on four complex regression problems: head pose estimation, facial landmark detection, age estimation, and end-to-end autonomous driving.
Revised: We evaluate BEL on four complex regression problems: head pose estimation, facial landmark detection, age estimation, and end-to-end autonomous driving.

Original: We make the following contributions in this work
Revised: We make the following contributions in this work:

Original: : • We propose binary-encoded labels for regression and introduce a general framework and a taxonomy for the design aspects of regression by binary classiﬁcation.
Revised: • We propose binary-encoded labels for regression and introduce a general framework and a taxonomy for the design aspects of regression by binary classiﬁcation.

Original: We propose desirable properties of encoding and decoding functions suitable for regression problems.
Revised: We propose desirable properties of encoding and decoding functions suitable for regression problems.

Original: • We present a series of suitable encoding, decoding, and loss functions for regression with BEL.
Revised: • We present a series of suitable encoding, decoding, and loss functions for regression with BEL.

Original: We present an end-to-end learning approach and regression layer architecture for BEL.
Revised: We present an end-to-end learning approach and regression layer architecture for BEL.

Original: We combine BEL with task-speciﬁc feature extractors for four tasks and evaluate multiple encoding, decoding, and loss functions.
Revised: We combine BEL with task-speciﬁc feature extractors for four tasks and evaluate multiple encoding, decoding, and loss functions.

Original: BEL outperforms direct regression for all the problems and specialized approaches for several tasks.
Revised: BEL outperforms direct regression for all the problems and specialized approaches for several tasks.

Original: • We theoretically and empirically demonstrate the effect of different design parameters on the accuracy, how it varies across different tasks, datasets, and network architectures, and provide preliminary insights and motivation for further study
Revised: • We theoretically and empirically demonstrate the effect of different design parameters on the accuracy, how it varies across different tasks, datasets, and network architectures, and provide preliminary insights and motivation for further study.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Binary classiﬁcation for regression: Prior works have proposed binary classiﬁcation-based approaches for ordinal regression (Crammer & Singer, 2001; Chu & Keerthi, 2005; Li & Lin, 2006).
Revised: Binary classiﬁcation for regression: Prior works have proposed binary classiﬁcation-based approaches for ordinal regression (Crammer & Singer, 2001; Chu & Keerthi, 2005; Li & Lin, 2006).

Original: Ordinal regression is a class of supervised learning problems, where the samples are labeled by a rank that belongs to an ordinal scale.
Revised: Ordinal regression is a class of supervised learning problems, where the samples are labeled by a rank that belongs to an ordinal scale.

Original: Ordinal regression approaches can be applied to regression by discretizing the numeric range of the real-valued labels (Fu et al., 2018; Berg etal. ; Niu et al., 2016).
Revised: Ordinal regression approaches can be applied to regression by discretizing the numeric range of the real-valued labels (Fu et al., 2018; Berg et al., 2021).

Original: In the existing works on ordinal regression by binary classiﬁcation, N − 1 binary classiﬁers are used for target labels ∈ { 1 , 2 , ..., N } , where classiﬁer- k predicts if the label is greater than k or not for a given input.
Revised: In the existing works on ordinal regression by binary classiﬁcation, N − 1 binary classiﬁers are used for target labels ∈ { 1 , 2 , ..., N } , where classiﬁer- k predicts if the label is greater than k or not for a given input.

Original: Li & Lin (2006) provided a reduction framework and generalization bound for the same.
Revised: Li & Lin (2006) provided a reduction framework and generalization bound for the same.

Original: However, the proposed binary classiﬁcation formulation is restricted.
Revised: However, the proposed binary classiﬁcation formulation is restricted.

Original: It requires several binary classiﬁers if the numeric range of output is extensive, whereas reducing the number of classiﬁers by using fewer quantization levels increases quantization error.
Revised: It requires several binary classiﬁers if the numeric range of output is extensive, whereas reducing the number of classiﬁers by using fewer quantization levels increases quantization error.

Original: Thus, a more generalized approach for using binary classiﬁcation for regression is desirable to allow ﬂexibility in the design of classiﬁers.
Revised: Thus, a more generalized approach for using binary classiﬁcation for regression is desirable to allow ﬂexibility in the design of classiﬁers.

Original: Binary classiﬁcation for multiclass classiﬁcation: Dietterich & Bakiri (1995) proposed the use of error-correcting output codes (ECOC) to convert a multiclass classiﬁcation to a set of binary classiﬁcation sub-problems.
Revised: Binary classiﬁcation for multiclass classiﬁcation: Dietterich & Bakiri (1995) proposed the use of error-correcting output codes (ECOC) to convert a multiclass classiﬁcation to a set of binary classiﬁcation problems.

Original: This improvesthe accuracyof the network as it introduces tolerance to binary classiﬁers’ errors depending upon the hamming distance (i.e., number of bits changed between two binary strings) between two codes.
Revised: This improves accuracy as it introduces tolerance to binary classiﬁers’ errors depending upon the hamming distance (i.e., number of bits changed between two binary strings) between two codes.

Original: Allwein et al.
Revised: Allwein et al.

Original: (2001) provided a unifying framework and multiclass loss bounds in terms of binary classiﬁcation loss.
Revised: (2001) provided a unifying framework and multiclass loss bounds in terms of binary classiﬁcation loss.

Original: More recent works have also used Hadamard code, a widely used error-correcting code (Song et al., 2021; Verma & Swami, 2019).
Revised: More recent works have also used Hadamard code, a widely used error-correcting code (Song et al., 2021; Verma & Swami, 2019).

Original: Other works have focused on the problem of extreme classiﬁcation and the useof ECOC and  compact codes (sublinear increase in the length of codes with the number of classes) for a classiﬁcation problem with a large number of classes (Cissé et al., 2012; Evron et al., 2018).
Revised: Other works have focused on [Figure] the use and design of compact codes that exhibit a sublinear increase in the length of codes with the number of classes for extreme classiﬁcation problems with a large number of classes (Cissé et al., 2012; Evron et al., 2018).

Original: However, the proposed encoding and decoding approaches do not consider the  ordinalityof labels for regression.Further,
Revised: However, the proposed encoding and decoding approaches do not consider the task objective and labels’ ordinality for regression.

Original: [Figure] the binary classiﬁers possess distinct error probability distribution properties for regression problems as observed empirically (Section 3.1), which can be exploited to design codes suitable for regression.
Revised: Further, the binary classiﬁers possess distinct error probability distribution properties for regression problems as observed empirically (Section 3.1), which can be exploited to design codes suitable for regression.

Original: Multiclass classiﬁcation and ordinal regression by binary classiﬁcation can be viewed as special cases falling under the BEL framework.
Revised: Multiclass classiﬁcation and ordinal regression by binary classiﬁcation can be viewed as special cases falling under the BEL framework.

Original: As shown in Section 4, other BEL designs yield improvements in accuracy over these approaches.
Revised: As shown in Section 4, other BEL designs yield improvements in accuracy over these approaches.

Original: Task-speciﬁc regression techniques are well explored as summarized below (see also Appendix D.
Revised: Task-speciﬁc regression techniques are well explored as summarized below (see also Appendix D).

Original: While effective, task-speciﬁc approaches lack generality by design.
Revised: While effective, task-speciﬁc approaches lack generality by design.

Original: Head pose estimation:
Revised: Head pose estimation:

Original: SSR-Net (Rothe et al., 2015) and FSA-Net (Yang et al., 2019) used softmax to reﬁne labels.
Revised: SSR-Net (Yang et al., 2018) and FSA-Net (fsa, 2019) used a soft stagewise regression approach.

Original: HopeNet (Ruiz et al., 2018) used a combination of classiﬁcation and regression loss.
Revised: HopeNet (Ruiz et al., 2018) used a combination of classiﬁcation and regression loss.

Original: QuatNet (Hsu et al., 2019) used a combination of regression and ordinal regression loss.
Revised: Hsu et al. (2019) used a combination of regression and ordinal regression loss.

Original: Facial landmark detection: Wang et al.
Revised: Facial landmark detection: Wang et al.

Original: (2020) minimize L2 loss between predicted and target 2D heatmaps with the latter formed using small variance Gaussians centered on ground truth landmarks.
Revised: (2020) minimize L2 loss between predicted and target 2D heatmaps with the latter formed using small variance Gaussians centered on ground truth landmarks.

Original: AWing (Wang et al., 2019) modiﬁed loss for different pixels in the heatmap.
Revised: AWing (Wang et al., 2019) modiﬁed loss for different pixels in the heatmap.

Original: LUVLi (Kumar et al., 2020) proposed a landmark’s location, uncertainty, and visibility likelihood-based loss.
Revised: LUVLi (Kumar et al., 2020) proposed a landmark’s location, uncertainty, and visibility likelihood-based loss.

Original: Bulat &
Revised: Bulat &

Original: Tzimiropoulos (2016) used binary heatmaps with pixel-wise binary cross-entropy loss.
Revised: Tzimiropoulos (2016) used binary heatmaps with pixel-wise binary cross-entropy loss.

Original: Age estimation: OR-CNN (Niu et al., 2016) and CORAL-CNN (Cao et al., 2020) used ordinal regression via binary classiﬁcation.
Revised: Age estimation: OR-CNN (Niu et al., 2016) and CORAL-CNN (Cao et al., 2020) used ordinal regression via binary classiﬁcation.

Original: MV-Loss (Pan et al., 2018) proposed to penalize the model output based on the age distribution
Revised: MV-Loss (Pan et al., 2018) proposed to penalize the model output based on the age distribution

Original: ’s variance, while Gao et al.
Revised: ’s variance, while Gao et al.

Original: (2018) proposed to use the KL-divergence between the softmax output and a generated label distribution for training.
Revised: (2018) proposed to use the KL-divergence between the softmax output and a generated label distribution for training.

Original:  B INARY - ENCODED L ABELS FOR R EGRESSION (BEL)
Revised:  B INARY - ENCODED L ABELS FOR R EGRESSION (BEL)

Original: We consider regression problems where the goal is to minimize the error between real-valued target labels y i and predicted labels ˆ y i , over a set of training samples i .
Revised: We consider regression problems where the goal is to minimize the error between real-valued target labels y i and predicted labels ˆ y i , over a set of training samples i .

Original: We transform this problem to a set of binary classiﬁcation sub-problems by converting a real-valued label to a binary code.
Revised: We transform this problem to a set of binary classiﬁcation sub-problems by converting a real-valued label to a binary code.

Original: Figure 1 shows the training and inference ﬂow for BEL.
Revised: Figure 1 shows the training and inference ﬂow for BEL.

Original: The red-colored blocks highlight functions that vary under BEL.
Revised: The red-colored blocks highlight functions that vary under BEL.

Original: A real-valued label y i ∈ R is quantized to level Q i ∈ { 1 , 2 , ..., N }  .
Revised: A real-valued label y i ∈ R is quantized to level Q i ∈ { 1 , 2 , ..., N } 1 (cid:13) .

Original: The quantized label isthen converted to a binary sequence B i ∈ { 0 , 1 } M  usingan encoding function E  .
Revised: The quantized label is converted to a binary vector B i ∈ { 0 , 1 } M , that we call a binary-encoded label , using encoding function E 2 (cid:13) .

Revised: There are (cid:0) 2 M N (cid:1) possible encoding functions—a large number.

Original: The binary-encoded labels B i are used to train M classiﬁers.
Revised: The binary-encoded labels B i are used to train M classiﬁers 3 (cid:13) .

Original: In the prediction phase, the  classiﬁers predict a binary code ˆ B i ∈ { 0 , 1 } M foran input x i  .
Revised: During inference the M classiﬁers predict a binary code ˆ B i ∈ { 0 , 1 } M for input x i 4 (cid:13) .

Original: The predictedbinary code ( ˆ B i ) or the real-valued conﬁdence vector of classiﬁers’ prediction ( ˆ Z i ) is then decoded to a predicted label ˆ y i ∈
Revised: The predicted code ( ˆ B i ) or the predictions’ magnitude ( ˆ Z i ), which indicates its conﬁdence, is then decoded to a predicted label ˆ y i ∈

Original: R using a decoding function D  .
Revised: R using a decoding function D 5 (cid:13) .

Original: We explore decoding functions that yield either quantized or continuous predicted outputs.
Revised: We explore decoding functions that yield either quantized or continuous predicted outputs.

Original: For the latter, we propose an expected correlation-based decoding function that decodes classiﬁers’ output to a continuous value, reducing the quantization error  (Section 3.3).
Revised: The latter avoids quantization error by employing expected correlation (Section 3.3).

Original: There are ﬁve key design parameters when applying BEL: quantization, encoding, decoding, regressor network architecture, and training loss formulation yielding a large design space.
Revised: BEL contains ﬁve major design parameters resulting in a large design space: quantization, encoding, decoding, regressor network architecture, and training loss formulation.

Original: In this work we consider only uniform quantization while leaving nonuniform quantization (Fu et al., 2018) to future work.
Revised: In this work we consider only uniform quantization while leaving nonuniform quantization (Fu et al., 2018) to future work.

Original: Section 3.2 and 3.3 explore the characteristics of suitable encoding, decoding, andtraining loss functions.
Revised: Section 3.2 and 3.3 explore the characteristics of suitable encoding, decoding, and loss functions.

Original: Section 3.4 explore the impact of regression network architecture.
Revised: Section 3.4 explores the impact of regressor network architecture.

Original: We ﬁnd varying any of these aspects can improve accuracy.
Revised: We ﬁnd varying any of these aspects can improve accuracy.

Original: While BEL provides a framework and some codes appear generally better than others, the most suitable BEL code to employappears to vary across task, dataset, and network architecture, as we show both theoretically (Section 3.1) and empirically (Section 4). 
Revised: While BEL provides a framework, and some design choices appear generally better than others, the most suitable BEL parameters to employ vary across task, dataset, and network architecture, as we show both theoretically (Section 3.1) and empirically (Section 4). [Table] [Figure]

Original: 3. A NALYSIS OF E NCODING /D ECODING F UNCTIONS
Revised: 3. A NALYSIS OF E NCODING /D ECODING F UNCTIONS

Original: This sectionanalytically analyzes the potential impact of encoding/decoding functions on regression error based upon empirically observed error distributions for the underlying classiﬁers.
Revised: This section analyzes the potential impact of encoding/decoding functions on regression error assuming empirically observed error distributions for the underlying classiﬁers.

Original: With this [Table] [Figure]
Revised: We compare Unary and Johnson codes (Figure 2b and 2c) to determine when each is preferable.

Original:  analytical study, we aim to understand the properties of classiﬁers for ordinal labels and their effect on regression error using simple  {E , D} functions and draw insights on desirable propertiesof these functions for regression and empirical results.
Revised: With this analytical study, we aim to obtain insight into ordinal label classiﬁer impact on regression error when employing simple encoding and decoding functions {E , D} . Based upon this analysis we identify desirable properties for these functions.

Original: We use unary (Figure 2b) and Johnson code (Figure 2c) for the analysis.

Original: The design of these codes and intuition for trying these codes are discussed in
Revised: The design of the codes and intuition for trying them are discussed in

Original: Section 3.2.
Revised: Section 3.2.

Original: We divide our analysis into three parts. First, the expected error of predicted labels is derived in terms of classiﬁers’ errors for two {E , D} functions.
Revised: We divide our analysis into three parts: First, the expected error of predicted labels is derived in terms of classiﬁers’ errors for two {E , D} functions.

Original: Next, we propose an approximate classiﬁer’s error probability distribution over the numeric range of target labels for regression based on empirical study.
Revised: Next, we propose an approximate classiﬁer’s error probability distribution over the numeric range of target labels for regression based on empirical study.

Original: Lastly, we compare the expected error of sample {E , D} functions based on our analysis.
Revised: Last, we compare the expected error of sample {E , D} functions based on our analysis.

Original: We use labels y i ∈ [1 , N − 1] , with quantization levels Q i ∈ { 1 , 2 , ..., N − 1 } .
Revised: We use labels y i ∈ [1 , N − 1] , with quantization levels Q i ∈ { 1 , 2 , ..., N − 1 } .

Original: Quantization error is not included as it is not affected by {E , D} functions.
Revised: Quantization error is not included as it is not affected by {E , D} functions

Original: Expected absolute error bounds in terms of classiﬁcation error:
Revised: Expected absolute error bounds in terms of classiﬁcation error:

Original: First, we analyze theexpected absolute error given the classiﬁcation error for a unary code, BEL-U.
Revised: First, we analyze the unary code (BEL-U).

Original: The encoding function E BEL-U converts Q i to B i =
Revised: The encoding function E BEL-U converts Q i to B i =

Original: b 1 i , b 2 i , ..., b N − 2 i , where b ki = 1 for k < Q i , else 0 .
Revised: b 1 i , b 2 i , ..., b N − 2 i , where b ki =for k < Q i , else 0 .

Original: A single-bit error changes the predicted value by one quantization level for a  decoding function that counts the number  in  the binary code. Hence the decoding function is deﬁned as:
Revised: In this case, a good choice of decoding function turns out to be simply counting the number of 1 outputs across all N − 2 classiﬁers since a error in a single classiﬁer changes the prediction by only one quantization level. Adding one since Q i = 1 is encoded by all zeros gives:

Original: [Equation]
Revised: [Equation]

Original: Let e k ( n ) be the error probability of classiﬁer k for target quantized label Q i = n .
Revised: Let e k ( n ) be the error probability of classiﬁer k for target quantized label Q i = n .

Original: For a uniform distribution of y i in the range [1 , N − 1], the expected error for BEL-U is:
Revised: For a uniform distribution of y i in the range [1 , N − 1] the expected error for BEL-U can be shown (see Appendix B) to be bounded as follows:

Original: [Equation]
Revised: [Equation]

Original: A similar analysis is appliedfor the Johnson code (referred to as BEL-J) in which Q i is encoded to a binary code B i 
Revised: A similar analysis of expected error can be applied to binary encoded labels constructed to yield Johnson codes (BEL-J), in which Q i is encoded using B i =

Original: = b 1 i , b 2 i , ..., b N/ 2 i , where, b ki = 1 for N 2 − Q i < k  (cid:54) N − Q
Revised: b 1 i , b 2 i , ..., b N/ 2 i , where, b ki = 1 for N 2 − Q i < k − 1 (cid:54) N − Q

Original: i , else 0 (Appendix B).
Revised: i , else 0 (see Equation 27 in Appendix B).

Original: Error probability of classiﬁers:
Revised: Error probability of classiﬁers:

Revised: To use Equation 2 we need to determine e k ( n ) .

Original: A classiﬁer’s target output is 0 or 1 for quantized target labelson an ordinal scale, and consists of  one or more bit transitions . The bit transition is a 0 → 1 or 1 →transition in theclassiﬁer’s target output over the numeric range of target labels.
Revised: A classiﬁer’s target output is 0 or 1 . For BEL, the target labels of a given classiﬁer will have one or more bit transitions from 0 → 1 or 1 → 0 as the target value of the regression network’s output varies.

Original: For example, for the unary code (Figure 2b), the target output of classiﬁer2 ( b 2 ) changes from 1 to 0 at a bit transition between Q i = 2 and Q i = 3 .
Revised: For example, for the unary code (Figure 2b), the target output of classiﬁer b 2 has a bit transition from 0 to 1 going from Q i = 2 to Q i = 3 .

Original: This classiﬁer should learn a decision boundarybetween targets in [1 , 2 . 5] and [2 . 5 , 8] .
Revised: The classiﬁer should learn a decision boundary in (2 , 3) .

Original: Thus, ideally, a classiﬁer learns a decision boundary for all bit transitions, and the complexity of the decision boundary increases with the number of bit transitions.
Revised: Each BEL classiﬁer is tasked with learning decision boundaries for all bit transitions. As the difﬁculty of this task varies with the number of bit transitions it varies with different encoding functions.

Original: It is known that the error/misclassiﬁcation probability of a classiﬁer tends to increase as the target label is closer to the  classiﬁer’s decision boundaries (Cardoso & Pinto da Costa, 2007).
Revised: Moreover, the misclassiﬁcation probability of a classiﬁer tends to increase as the target label is closer to the [Figure] classiﬁer’s decision boundaries (Cardoso & Pinto da Costa, 2007).

Original: Thus, we approximate e k ( y ) for a classiﬁer k with t bit transitions as a linear combination of t Gaussian distributions.
Revised: Thus, we approximate e k ( y ) for a classiﬁer k with t bit transitions as a linear combination of t Gaussian distributions.

Original: Here, each
Revised: Here, each

Original: Gaussian term is centered around a bit transition.
Revised: Gaussian term is centered around a bit transition.

Original: Let f N ( µ,σ 2 ) ( y ) denote the probability density of a normal distribution with mean µ and variance σ 2 .
Revised: Let f N ( µ,σ 2 ) ( y ) denote the probability density of a normal distribution with mean µ and variance σ 2 .

Original: Each classiﬁer for BEL-U encoding has one bit transition, whereas, each classiﬁer for BEL-J encoding has two bit transitions (except the ﬁrst and last classiﬁers).
Revised: Each classiﬁer for BEL-U encoding has one bit transition, whereas, each classiﬁer for BEL-J encoding has two bit transitions (except the ﬁrst and last classiﬁers).

Original: e k ( y ) of a classiﬁer k for BEL-U and BEL-J encoding is approximated as: [Equation] y 
Revised: e k ( y ) of a classiﬁer k for BEL-U and BEL-J encoding is approximated as: e BEL-U k ( y ) =

Original: [Equation] where, µ k = k + 0 .(3)[Figure] [Figure]
Revised: rf N ( µ k ,σ 2 ) ( y ) , where, µ k = k + 0 .(3)

Original: [Equation]
Revised: e BEL-J k ( y ) =

Original: [Figure] [Equation]
Revised: rf N ( µ 1 k ,σ 2 ) ( y ) + rf N ( µ 2 k ,σ 2 ) ( y ) , where, µ 1 k = N 2 − k + 1 . 5 , µ 2 k = N − k + 1 .(4)

Original: Here, r is a scaling factor for the error probabilities.
Revised: Here, r is a scaling factor.

Original: Figure 3a andFigure 3b show the empirically observedand proposed approximate errorprobability distributions for two classiﬁerstrained using  HRNetV2-W18 (Wang et al., 2020) and COFW facial landmark detection dataset (Burgos-Artizzu et al., 2013) for labels y ∈ [1 , 256] .
Revised: Figure 3a and 3b compares Equation 3 and 4 against empirically observed error distributions for two classiﬁers using an HRNetV2-W18 (Wang et al., 2020) feature extractor (backbone) trained with the COFW facial landmark detection dataset (Burgos-Artizzu et al., 2013).

Original: Comparison of expected absolute error for BEL-U and BEL-J:
Revised: Comparison of expected absolute error for BEL-U and BEL-J:

Original: Based on equation 2equation 4, we compare  expected absolute errors of BEL-U and BEL-J encoding/decoding functions. Figure 3c represents the percentage increase in absolute error for BEL-U compared to BEL-J for valid values of variance σ2 ( y − axis) and scaling factor r ( x − axis) from equation 3 andequation 4.
Revised: Based on the above analysis, we compare the expected absolute errors of BEL-U and BEL-J. Figure 3c represents the percentage increase in absolute error for BEL-U compared to BEL-J for valid values of standard deviation σ ( y − axis) and scaling factor r ( x − axis) as used in Equation 3 and 4.

Original: Here, BEL-J has a lower error in the red-colored region (%increase > 0 ), whereas BEL-U has a lower error in the blue-colored region (%increase < 0 ).
Revised: Here, BEL-J has a lower error in the red-colored region (%increase > 0 ), whereas BEL-U has a lower error in the blue-colored region (%increase < 0 ).

Original: The ﬁgure shows that  BEL-J or BEL-U has lower error depending upon the values  σ and r used to approximate the classiﬁers’ error probability distribution.
Revised: The ﬁgure shows that whether BEL-J or BEL-U has lower error depends upon the values of σ and r .

Original: We leave the derivation of these parameters from the ﬁrst principles to future work.

Original: Further theoretical study of the properties of classiﬁers for ordinal labels can provide bounds on these values, which will also help with encoding/decoding design.

Original: Thus, the choice of {E , D} functions heavily depends upon the classiﬁers’ error probability distribution, which depends uponmany factors, including the task, dataset, label distribution, network architecture, and optimization approach.
Revised: This suggests that the best {E , D} function will depend upon the classiﬁer error probability distribution. The classiﬁer error distribution in turn may depend upon the task, dataset, label distribution, network architecture, and optimization approach.

Original: We provide the derivation of expected error for BEL-U and BEL-J and classiﬁers’ empirical error probability distributions for different architectures, datasets, and encodings  in Appendix B-C.
Revised: Derivation of expected error for BEL-U and BEL-J and classiﬁers’ empirical error probability distributions for different architectures, datasets, and encodings are provided in Appendix B to C.

Original: 3. D ESIGN OF E NCODING F UNCTIONS
Revised: 3. D ESIGN OF E NCODING F UNCTIONS

Original: The distance between two codes (e.g., Hamming distance) gives a measure of error correction capability. The objective of regression is to minimize the absolute/squared error between predicted and target values.
Revised: Based on the above analysis and further empirical observation we identify three principles for selecting BEL codes for regression so as to minimize error.

Revised: First, individual classiﬁers should require fewer bit transitions as this makes them easier to train.

Original: The encoding function for BEL can be designed to take advantage of this fact. Thus, a desirable property for a BEL encoding function is that: the  distance between two quantized values should be proportional to the distance between their codes.
Revised: Second, a desirable property for a BEL encoding function is that the hamming distance between two codes (number of bits that differ) should be proportional to the difference between the target values they encode. However, hamming distance weighs all bit changes equally.

Original: Further, the error/misclassiﬁcation probability of classiﬁers tends to increase as the target gets closer to a bit transition (Figure 3a-3b). Since hamming distance (i.e., number of bits changed between two codes) weighs all the bits equally, hamming distance-based code design provides equal error protection capability to allthe bits (Wu, 2018; Xie et al., 2002) and does not account for which classiﬁers are more likely to mispredict for  given  target value.
Revised: Thus, hamming distance based code design provides equal error protection capability to all bits (Wu, 2018; Xie et al., 2002) and does not account for which classiﬁers are more likely to mispredict for a given input. This matters because the misclassiﬁcation probability of BEL classiﬁers is not uniform, but rather increases the closer the target value of an input is to a bit transition (e.g., Figure 3a and 3b).

Original: This gives the second property of a suitable encoding: the distance metric must consider how likely each classiﬁer is to mispredict for some target value  toaccount for classiﬁers’ nonuniform error probability distribution for aregression problem. This design property covers the impact of bit transitions in codes on continuous predictions.
Revised: These observations yield a third important consideration: For a given target value classiﬁers closer to a bit transition are more likely to incur an error. The principles above highlight a tradeoff between classiﬁcation error probability and error-correction properties when selecting BEL codes.

Original: We evaluateseveral encodings that to greater or lesser extent exhibit these desirable properties. We focus on encodings that reduce the number of classiﬁers (bits) while balancing the separation between the bit transitions per classiﬁer.
Revised: To evaluate the trade-offs, we empirically evaluate encodings that, to greater or lesser extent, satisfy one or more of the principles while focusing on reducing the number of classiﬁers (bits) so as to avoid increasing model parameters.

Original: Below we empirically evaluate the impact of these different encodings on previously proposed deep regression networks.

Original: We leave the development of procedures for automatically discovering optimized encodings to future work.
Revised: Development of algorithms that might systematically optimize encoding functions is left to future work.

Original: Figure 2 shows example codes for encodings explored.

Original: We explain all encodings in detail below.
Revised: Speciﬁcally, we explore the following codes:

Original: Unary code (U): The unary code (Figure 2b) is a simple code with one bit transition per classiﬁer and requires N − 1 bits to encode N values.
Revised: Unary code (U): Unary codes (Figure 2b) have only one bit transition per classiﬁer and thus require M = N − 1 bits to encode N values.

Original: Prior works on ordinal regression by binary classiﬁcation (Li & Lin, 2006; Niu et al., 2016) usesa similar formulation of classiﬁers as the unary code.
Revised: Unary codes satisfy the ﬁrst two principles and prior work on ordinal regression by binary classiﬁcation (Li & Lin, 2006; Niu et al., 2016) uses similar codes.

Original: Johnson code (J): The Johnson code sequence (Figure 2c) is based on Libaw-Craig code (Libaw &
Revised: Johnson code (J): The Johnson code sequence (Figure 2c) is based on Libaw-Craig code (Libaw &

Original: Craig, 1953).
Revised: Craig, 1953).

Original: We select this code as it has well-separated bit transitions and requires  N 2 bits compared to N required for unary code.
Revised: We select this code as it has well-separated bit transitions and requires M = N 2 bits compared to N required for unary code.

Original: This code exempliﬁes the difference between hamming distance  and a distance that considers classiﬁers’ error probability distribution.
Revised: This code exempliﬁes the impact of considering non-uniform classiﬁer error probabilities (third principle). For example the hamming distance between 1 and 8 is just one.

Original: Although the hamming distance between 1 and 7 is small, since the differing bits ( C 1 and C 4 ) are far from  bit transitions and less likely to be mispredicted, a 1 is less likely to be mispredicted as a 7 , reducing the regression error.
Revised: However, the bit transition for the differing bit, for classiﬁer b 1 , is far from 1 or 8 . Assuming equal error probability distributions centered on each bit transition for each classiﬁer (as in Equation 4), b 1 is less likely to mispredict than b 2 , b 3 or b 4 for inputs with target values near 1 or 8 .

Original: Base+displacement based code (B1JDJ/B2JDJ): We further reduce the number of bits using a base+displacement-based representation.
Revised: Base+displacement based code (B1JDJ/B2JDJ): We further reduce the number of bits using a base+displacement-based representation.

Original: In this representation, a value is represented in base-k using a radix-term r and displacement d via r * k + d . r and d are represented using Johnson codes.
Revised: In this representation, a value is represented in base-k using a base-term b and displacement d via b * k + d . b and d are represented using Johnson codes.

Original: Further, to improve the distance between two remote codes, we adapt reﬂected binary codes for term d (Gray, 1953).
Revised: Further, to improve the distance between two remote codes, we adapt reﬂected binary codes for term d (Gray, 1953).

Original: We evaluate base-2 (B1JDJ - Figure 2e) and base-4 codes (B2JDJ - Figure 2f).
Revised: We evaluate base-2 (B1JDJ - Figure 2e) and base-4 codes (B2JDJ - Figure 2f).

Original: Binary coded hex - Johnson code (HEXJ): In HEXJ, each digit (0-F) of the hexadecimal representation of a number is converted to an 8-bit binary code using Johnson code.
Revised: Binary coded hex - Johnson code (HEXJ): In HEXJ (Figure 2g), each digit (0-F) of the hexadecimal representation of a number is converted to an 8-bit binary code using Johnson code.

Original: For example, for the decimal number 47 (i.e., 2F in hex), HEXJ(47) = Concetanate(Johnson(2), Johnson(F)).
Revised: For example, for the decimal number 47 (i.e., 2F in hex), HEXJ(47) = Concetanate(Johnson(2), Johnson(F)).

Original: A 16-bit HEXJ code can represent numbers in the range of 00 to FF (a total of 256).
Revised: A 16-bit HEXJ code can represent numbers in the range of 00 to FF (a total of 256).

Original: The number of bits increases sublinearly with the number of quantization levels for HEXJ, making it suitable for regression problems with many quantization levels.
Revised: The number of bits increases sublinearly with the number of quantization levels for HEXJ, making it suitable for regression problems with many quantization levels.

Original: Hadamard code (HAD): Hadamard codes (Bose & Shrikhande, 1959) are widely used as errorcorrecting codesdue to their fault-tolerant properties andalso have been used to encode classes in multiclass classiﬁcation (Dietterich & Bakiri, 1995; Verma & Swami, 2019).
Revised: Hadamard code (HAD): Hadamard codes (Bose & Shrikhande, 1959) are widely used as errorcorrecting codes and have been used for multiclass classiﬁcation (Dietterich & Bakiri, 1995; Verma & Swami, 2019).

Original: It requires N bits to encode N values.
Revised: They require M = N bits to encode N values.

Original: However,in the Hadamard code, each column has many bit transitions, increasing the complexity of the decision boundary to be learned by eachbinary classiﬁer. Further, the Hadamard code does not follow the desirable properties proposed in this work for regression tasks.
Revised: However, Hadamard codes violate all three BEL code selection principles: First, each classiﬁer has many bit transitions. Second, as each code is equidistant (hamming distance of M 2 ), the difference between target values is ignored. Finally, they protect all bits equally so do not take advantage of non-uniform error probabilities.

Original: We evaluate the Hadamard code to show that it is not suitable for regression (Section A).
Revised: We verify empirically Hadamard codes are unsuitable for regression (Section A).

Original: 3. D ESIGN OF D ECODING F UNCTIONS
Revised: 3. D ESIGN OF D ECODING F UNCTIONS

Original: We explore three decoding functions: custom decoding, correlation-based decoding, and expected correlation-based decoding.
Revised: We explore three decoding functions: custom decoding, correlation-based decoding, and expected correlation-based decoding.

Original: Custom decoding functions are speciﬁc to the encoding function, and are only evaluated for unary and Johnson codes.
Revised: Custom decoding functions are speciﬁc to the encoding function, and are only evaluated for unary and Johnson codes.

Original: In contrast, correlation-based decoding, ﬁrst explored in prior work studying ECOC for multiclass classiﬁcation (Allwein et al., 2001), can be applied to all codes.
Revised: In contrast, correlation-based decoding, ﬁrst explored in prior work studying ECOC for multiclass classiﬁcation (Allwein et al., 2001), can be applied to all codes.

Original: For  labels in { 1 , 2 , ..., N } , we deﬁne a code matrix C of size N × M , where M is the number of bits/classiﬁers used for the binary-encoded label.
Revised: For quantized labels in { 1 , 2 , ..., N } , we deﬁne a code matrix C of size N × M , where M is the number of bits/classiﬁers used for the binary-encoded label.

Original: Each row C k, : in this matrix represents the binary code for label Q i = k .
Revised: Each row C k, : in this matrix represents the binary code for label Q i = k .

Original: For example, Figure 2b can be considered a code matrix, where the ﬁrst row represents code for label Q i = 1 .
Revised: For example, Figure 2b can be considered a code matrix, where the ﬁrst row represents code for label Q i = 1 .

Original: Let ˆ Z
Revised: Let ˆ Z

Original: i ∈
Revised: i ∈

Original: R M denote the output logit values of the classiﬁers.
Revised: R M denote the output logit values of the classiﬁers.

Original: For decoding, the row with the highest correlation with the output ˆ
Revised: For decoding, the row with the highest correlation with the output ˆ

Original: Z i is selected as the decoded label.
Revised: Z i is selected as the decoded label.

Original: Output logit value vector ˆ Z i is used instead of output binary code ˆ B i to ﬁnd the correlation as it uses the conﬁdence of a classiﬁer to make a more accurate prediction.
Revised: Here, real-valued output ˆ Z i is used instead of output binary code ˆ B i to ﬁnd the correlation as it uses the conﬁdence of a classiﬁer to make a more accurate prediction.

Original: For target quantized labels Q i ∈ { 1 , 2 , ..., N } , the decoding function is deﬁned as: 
Revised: For target quantized labels Q i ∈ { 1 , 2 , ..., N } , the decoding function is deﬁned as: [Equation]

Original: [Equation]
Revised: [Equation]

Original: However, D GEN outputs a quantized prediction, introducing quantization error.
Revised: However, D GEN outputs a quantized prediction, introducing quantization error.

Original: To remedy this concern and demonstrate the potential of more sophisticated decoding rules, we propose and evaluate an expected correlation-based decoding function, which allows prediction of real-valued label ˆ y i .
Revised: To remedy this concern and demonstrate the potential of more sophisticated decoding rules, we propose and evaluate an expected correlation-based decoding function, which allows prediction of real-valued label ˆ y i .

Original: For target labels y i ∈ [1 , N ] , the decoding function is deﬁned as:
Revised: For target labels y i ∈ [1 , N ] , the decoding function is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original:  where 
Revised: k , where σ k =

Original: [Equation]
Revised: [Equation]

Original: A deep neural network with multiple output binary classiﬁers can be trained using the binary cross-entropy (BCE) loss L BCE (cid:0) ˆ Zi , E ( Q i ) (cid:1) .
Revised: ˆ Z

Original: However, this loss minimizes [Figure] [Table] [Figure]
Revised: [Equation]

Original: [Table] [Figure] [Table] [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure] [Table] [Figure] [Table]
Revised: Training loss functions:

Original: [Figure] [Table] [Figure] [Table]
Revised: A deep neural network with multiple output binary classiﬁers can be trained using the binary cross-entropy (BCE) loss L BCE (cid:0)

Original: [Figure] [Table] [Figure] [Table]
Revised: ˆ Z i , E ( Q i ) (cid:1) .

Original: [Figure] [Table] [Figure] the mismatch between predicted and target code but does not directly minimize the error between the target and predicted values.
Revised: However, this loss minimizes the mismatch between predicted and target code but does not directly minimize the error between the target and predicted values.

Original: Decoding functions D GEN and D GEN-EX can be used to calculate the loss and minimize the mismatch between decoded predictions and target values directly.
Revised: Decoding functions D GEN and D GEN-EX can be used to calculate the loss and minimize the mismatch between decoded predictions and target values directly.

Original: Decoding function D GEN ﬁnds the correlation between each row of the code matrix ( C k, : ) and the output ˆ Z i .
Revised: Decoding function D GEN ﬁnds the correlation between each row of the code matrix ( C k, : ) and the output ˆ Z i .

Original: C ˆ Z i gives the correlation vector, and the index with the highest correlation is used as the predicted label.
Revised: C ˆ Z i gives the correlation vector, and the index with the highest correlation is used as the predicted label.

Original: In this case, cross-entropy loss L CE (cid:0) C
Revised: In this case, cross-entropy loss L CE (cid:0) C

Original: ˆ Z i , Q i (cid:1) can be used to train the network.
Revised: ˆ Z i , Q i (cid:1) can be used to train the network.

Original: Similarly, for decoding function D GEN-EX , which predicts a continuous value, L1 or L2 loss L L1/L2 (cid:0) D GEN-EX ( ˆ Z i ) , y i (cid:1) can also be used for training.
Revised: Similarly, for decoding function D GEN-EX , which predicts a continuous value, L1 or L2 loss L L1/L2 (cid:0) D GEN-EX ( ˆ Z i ) , y i (cid:1) can also be used for training.

Original: We evaluate multiple combinations of decoding and loss functions in Section 4.
Revised: We evaluate multiple combinations of decoding and loss functions in Section 4.

Original: 3. R EGRESSION N ETWORK A RCHITECTURE FOR BEL
Revised: 3. R EGRESSION N ETWORK A RCHITECTURE FOR BEL

Original: Adeep regression network typically consists of a feature extractor and regressor.
Revised: A regression network typically consists of a feature extractor and regressor.

Original: The regressor consists of a fully connected layer between the feature extractor’ output (i.e., feature vector) and output logits. Figure 4a represents a network architecture for direct regression.
Revised: The regressor consists of a fully connected layer between the feature extractor’s output (i.e., feature vector) and output logits for direct regression as shown in Figure 4a.

Original: In BEL, the number of output logits is increased to the number of classiﬁers (bits) used.
Revised: In BEL, the number of output logits is increased to the number of classiﬁers (bits) used.

Original: The size of the fully connected regression layer increases with  the  number of output logits.
Revised: When y ∈ R P , with P > 1 , the required number of output logits— P × M assuming M -bit BEL encoding per output dimension—can signiﬁcantly increase the size of the regression layer.

Original: However, empirically, we ﬁnd small feature vectors sufﬁce as the output logits are highly correlated for the  encoding functions we explore.
Revised: However, empirically, we ﬁnd small feature vectors sufﬁce as the output logits are highly correlated for the explored encoding functions.

Original: Adding a fully connected  layer to reduce feature vector size to θ reduces the number of parameters and provide a trade-off between the model size and accuracy.
Revised: Adding a fully connected bottleneck layer to reduce feature vector size to θ reduces the number of parameters and provides a trade-off between the model size and accuracy.

Original: Figure 4b shows the modiﬁedneural network architecture for BEL.
Revised: Figure 4b shows the modiﬁed network architecture for BEL.

Original:  E VALUATION
Revised:  E VALUATION

Original: Table 1 summarizes the tasks, datasets, and network architectures used for the evaluation of BEL.
Revised: Table 1 summarizes the tasks, datasets, and network architectures used for the evaluation of BEL.

Original: These tasks are commonly used for evaluation of regression approaches by prior works due to the complexity of problem and network architectures (Díaz & Marathe, 2019).
Revised: These tasks are commonly used for evaluation of regression approaches by prior works due to the complexity of problem and network architectures (Díaz & Marathe, 2019).

Original: Landmark-free 2D head pose estimation (HPE) aims to ﬁnd a human head’s pose in terms of three angles: yaw, pitch, and roll from a 2D image without landmarks.
Revised: Landmark-free 2D head pose estimation (HPE) aims to ﬁnd a human head’s pose in terms of three angles: yaw, pitch, and roll from a 2D image without landmarks.

Original: Facial landmark detection (FLD) is a problem of detecting the ( x, y ) coordinates of keypoints in a given face image.
Revised: Facial landmark detection (FLD) is a problem of detecting the ( x, y ) coordinates of keypoints in a given face image.

Original: Age estimation aims to predict the age of a person from an image.
Revised: Age estimation aims to predict the age of a person from an image.

Original: In end-to-end autonomous driving steering 
Revised: In end-to-end autonomous driving, the steering wheel’s next angle is predicted

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: rr
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: ( M
Revised: / N

Original: [Figure]
Revised: [Figure]

Original: / N M E )

Original: HPE
Revised: HPE

Original: D 1 D 2 D

Original: [Figure]

Original: D
Revised: D

Original: [Figure]
Revised: [Figure]

Original: D
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: HPE
Revised: HPE

Original: D 1 D2 D 3 D 4 D
Revised: D 1 D

Original: [Figure]
Revised: [Figure]

Original: HPE
Revised: HPE

Original: D 1 D

Original: [Figure]
Revised: [Figure]

Original: HPE
Revised: HPE

Original: D

Original: [Figure]
Revised: [Figure]

Original: D

Original: [Figure]
Revised: [Figure]

Original: D

Original: [Figure] [Figure][Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: FLD
Revised: FLD

Original: D

Original: [Figure]
Revised: [Figure]

Original: FLD
Revised: FLD

Original: [Figure]
Revised: [Figure]

Original: ( M A E / N M E )

Original: FLD

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: FLD

Original: [Figure]
Revised: [Figure]

Original: AE
Revised: AE

Original: [Figure]
Revised: [Figure]

Original: AE
Revised: AE

Original: [Figure]
Revised: [Figure]

Original: PN
Revised: PN

Original: U
Revised: U

Original: J
Revised: J

Original: B1JDJ
Revised: B1JDJ

Original: B2JDJ HEXJ 
Revised: [Figure] HEXJ from an image of the road.

Original: The feature extractor, data augmentation, evaluation protocols, and the number of training iterations are kept uniform across different methods for each benchmark.
Revised: Normalized Mean Error (NME) and Mean Absolute Error (MAE) with respect to raw real-valued labels are used as the evaluation metric for FLD and the rest, respectively.

Original: We also evaluate direct regression and multiclass classiﬁcation as baseline regression approaches.
Revised: We also evaluate direct regression and multiclass classiﬁcation as baseline regression approaches.

Original: For direct regression, L1 or L2 loss functions are used.
Revised: For direct regression, L1 or L2 loss functions are used.

Original: Label values are scaled to reduce the range of labels.
Revised: Label values are scaled to reduce the range of labels.

Original: The loss function and the scaling factors are set using hyperparameter tuning.
Revised: The loss function and the scaling factors are set using hyperparameter tuning.

Original: In the multiclass classiﬁcation-based regression, the target values are quantized and converted to a class.
Revised: In the multiclass classiﬁcation-based regression, the target values are quantized and converted to a class.

Original: The network is trained using cross-entropy loss in this case.
Revised: The network is trained using cross-entropy loss in this case.

Original: In our evaluation, the entire network (i.e., feature extractor and regressor) are trained end-to-end for direct regression, multiclass classiﬁcation, and BEL.
Revised: In our evaluation, the entire network (i.e., feature extractor and regressor) are trained end-to-end for direct regression, multiclass classiﬁcation, and BEL.

Revised: The feature extractor, data augmentation, evaluation protocols, and the number of training iterations are kept uniform across different methods for each benchmark.

Original: We report average of ﬁve training runs and error margin of 95% conﬁdence interval.
Revised: We report average of ﬁve training runs and error margin of 95% conﬁdence interval.

Original: Details on datasets, training parameters, related work for speciﬁc tasks, and other evaluation metrics are provided in Appendix C.
Revised: Details on datasets, training parameters, related work for speciﬁc tasks, and other evaluation metrics are provided in Appendix C.

Original: BEL introduces several design parameters for regression by binary classiﬁcation.
Revised: BEL introduces several design parameters for regression by binary classiﬁcation.

Original: We evaluate different encoding ( E ), decoding ( D ), and training loss ( L ) functions for BEL across all the benchmarks and study the extent and nature of the impact of these design parameters on accuracy.
Revised: We evaluate different encoding ( E ), decoding ( D ), and training loss ( L ) functions for BEL across all the benchmarks and study the extent and nature of the impact of these design parameters on accuracy.

Original: Encoding function ( E ): Figure 5 plots error (MAE or NME) using different encodings.
Revised: Encoding function ( E ): Figure 5 plots error (MAE or NME) using different encodings.

Original: We do not show results for Hadamard codes in the ﬁgure as it results in signiﬁcantly higher error than other encodings (Appendix A).
Revised: We do not show results for Hadamard codes here as it results in signiﬁcantly higher error than other encodings (Appendix A).

Original: On average, Hadamard codes result in  60% higher error than J encoding, which shows that these codes are unsuitable for regression.
Revised: On average, Hadamard codes result in ∼ 60% higher error than J encoding, which shows that these codes are unsuitable for regression.

Original: The results show the encoding function has a signiﬁcant impact on the accuracyof several benchmarks and the best-performing encoding function varies across tasks, datasets, and network architectures (e.g., HPE1 and HPE3 are trained on the same dataset and different architecture).
Revised: The results show the encoding function signiﬁcantly affects the accuracy and the best-performing encoding function varies across tasks, datasets, and network architectures (e.g., HPE1 and HPE3 are trained on the same dataset and different architecture).

Original: In Section 3.1 we observed that which encoding/decoding functions result in lower error depends upon the classiﬁers’ error distribution.
Revised: In Section 3.1 we observed that which encoding/decoding functions result in lower error depends upon the classiﬁers’ error distribution.

Original: For HPE3, FLD1, and AE1, J does better than U for D1 (decoding functions used for analytical comparison in Section 3.1); we attribute this to misclassiﬁcation errors occurring more frequently near bit transitionsin these tasks based on the analytical study.
Revised: For decoding functions used for the comparison in Section 3.1, J does better than U for HPE3, FLD1, and AE1; we attribute this to misclassiﬁcation errors occurring more frequently near bit transitions based on the analytical study.

Original: The encoding function impacts the number of classiﬁers and the complexity of the function to be learned by a classiﬁer.
Revised: The encoding function impacts the number of classiﬁers and the complexity of the function to be learned by a classiﬁer.

Original: We observe a trade-off between these two parameters.
Revised: We observe a trade-off between these two parameters.

Original: For some benchmarks, the availability of sufﬁcient training data and network capacity facilitates the learning of complex classiﬁers such as B2JDJ.
Revised: For some benchmarks, the availability of sufﬁcient training data and network capacity facilitates the learning of complex classiﬁers such as B2JDJ.

Original: In such a case, a reduced number of classiﬁers compared to U, J, or B1JDJ codes results in a lower error.
Revised: In such a case, a reduced number of classiﬁers compared to U, J, or B1JDJ codes results in a lower error.

Original: We provide empirical results for the same in Appendix A.
Revised: We provide empirical results for the same in Appendix A.

Original: Decoding ( D ) and training loss ( L ) functions: We explore three decoding  functions (Section 3.3).
Revised: Decoding ( D ) and training loss ( L ) functions: We explore three decoding and three training loss functions (Section 3.3).

Original: The training loss functions can be selected from BCE, CE, and L1/L2 loss.

Original: However, not all the combinations of decoding and loss functions  perform well.
Revised: However, not all the combinations of decoding and loss functions ( D / L ) perform well.

Original: For example, CE, L1, or L2 losses do not use  BEL-J or  BEL-U decodings.
Revised: For example, CE, L1, or L2 losses do not use decodings D BEL-J or D BEL-U .

Original: Therefore, optimizing the network for these losses does not directly minimize the absolute error between the target and decoded predictions.
Revised: Therefore, optimizing the network for these losses does not directly minimize the absolute error between targets

Original: In contrast, GENEX decoding is used for L1/L2 loss function calculation; in such a case, the training loss directly

Original: [Table]
Revised: [Table]

Original: minimizes the error between target values and decoded values.
Revised: and decoded predictions.

Original: Out of such nine combinations, we present results for ﬁve  combinations.
Revised: We present results for ﬁve out of nine D / L combinations.

Original: Figure 5 compares error (MAE or NME) achieved by different D / L combinations and highlights the range of error variations across D / L .
Revised: Figure 5 compares error (MAE or NME) achieved by different D / L combinations and highlights the range of error variations.

Original: D GEN-EX results in the lowest error for the majority of the benchmarks as it reduces quantization error and also takes advantage of the output logit conﬁdence values.
Revised: D GEN-EX results in the lowest error for the majority of the benchmarks as it reduces quantization error and also utilizes the output logit conﬁdence values.

Original: D GEN-EX consistently perform better than D GEN function that has been used for multiclass classiﬁcation by prior works (Allwein et al., 2001).
Revised: D GEN-EX consistently perform better than D GEN function that has been used for multiclass classiﬁcation by prior works (Allwein et al., 2001).

Original: The use of CE or L1/L2 loss results in a lower error with D GEN-EX for most benchmarks as the training loss function  minimizes the error between decoded labels and target labels.
Revised: The use of CE or L1/L2 loss results in a lower error with D GEN-EX for most benchmarks as the training loss function directly minimizes the error between targets and decoded predictions.

Original: Comparison of BEL with regression approaches:
Revised: Comparison of BEL with regression approaches:

Original: Table 2 compares BEL with other approaches for different benchmarks (Table 1).
Revised: Table 2 compares BEL with other approaches for different benchmarks (Table 1).

Original: We explore and evaluate multiple combinations of encoding ( E ), decoding ( D ), and loss ( L ) functions for BEL in this work.
Revised: We explore and evaluate multiple combinations of encoding ( E ), decoding ( D ), and loss ( L ) functions for BEL in this work.

Original: The 20% of the training set is used as validation set and the validation error is used to choose the best BEL approach.
Revised: In these experiments 20% of the training set is used as validation set and the validation error is used to choose the best BEL approach.

Original: An ablation study for using more fully connected layers for direct regression and multiclass classiﬁcation  in Appendix A. BEL results in lower error than direct regression and multiclass classiﬁcation and even outperforms task-speciﬁc regression approaches for several benchmarks.
Revised: An ablation study for using more fully connected layers for direct regression and multiclass classiﬁcation is in Appendix A. BEL results in lower error than direct regression and multiclass classiﬁcation and even outperforms task-speciﬁc regression approaches for several benchmarks.

Original: The results show no single combination of encoding/decoding/loss functions evaluated was best for all benchmarks but also demonstrate BEL improves accuracy across a range of regression problems.
Revised: The results show no single combination of encoding/decoding/loss functions evaluated was best for all benchmarks but also demonstrate BEL improves accuracy across a range of regression problems.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: This work proposes binary-encoded labels (BEL) to pose regression as binary classiﬁcation.
Revised: This work proposes binary-encoded labels (BEL) to pose regression as binary classiﬁcation.

Original: We propose a taxonomy identifying the key design aspects for regression by binary classiﬁcation and demonstrate the impact of classiﬁcation error and encoding/decoding functions on the expected label error.
Revised: We propose a taxonomy identifying the key design aspects for regression by binary classiﬁcation and demonstrate the impact of classiﬁcation error and encoding/decoding functions on the expected label error.

Original: Different encoding, decoding, and loss functions are explored to evaluate our approach using four complex regression tasks.
Revised: Different encoding, decoding, and loss functions are explored to evaluate our approach using four complex regression tasks.

Original: BEL results in an average 9 . 9% , 15 .
Revised: BEL results in an average 9 . 9% , 15 .

Original: 5% , and 7 . 2% lower error than direct regression, multiclass classiﬁcation, and task-speciﬁc regression approaches, respectively.
Revised: 5% , and 7 . 2% lower error than direct regression, multiclass classiﬁcation, and task-speciﬁc regression approaches, respectively.

Original: BEL improves accuracy over state-of-the-art approaches for head pose estimation (BIWI, AFLW2000), facial landmark detection (COFW), age estimation (AFAD), and end-to-end autonomous driving (PilotNet).
Revised: BEL improves accuracy over state-of-the-art approaches for head pose estimation (BIWI, AFLW2000), facial landmark detection (COFW), age estimation (AFAD), and end-to-end autonomous driving (PilotNet).

Original: Our analysis and empirical evaluation in this work demonstrate the potential of the vast design space of BEL for regression problems and the importance of ﬁnding suitable design parameters for a given task.
Revised: Our analysis and empirical evaluation in this work demonstrate the potential of the vast design space of BEL for regression problems and the importance of ﬁnding suitable design parameters for a given task.

Original: The best performing encoding/decoding function pair may be task, dataset, and network speciﬁc.
Revised: The best performing encoding/decoding function pair may be task, dataset, and network speciﬁc.

Original: A possibility this suggests, which we leave to future work, is that it may be beneﬁcial to develop automated approaches to optimizing these functions,
Revised: A possibility this suggests, which we leave to future work, is that it may be beneﬁcial to develop automated approaches for optimizing these functions.  A CKNOWLEDGEMENTS

Revised: This research has been funded in part by the National Sciences and Engineering Research Council of Canada (NSERC) Strategic Project Grant.

Revised: Tor M. Aamodt serves as a consultant for Huawei

Original:  Reproducibility:
Revised: Technologies Canada Co. Ltd. and Intel Corp. Deval Shah is partly funded by the Four Year Doctoral Fellowship (4YF) provided by the University of British Columbia. Reproducibility:

Original: We have provided a detailed discussion about training hyperparameters, experimental setup, and modiﬁcations made in publicly available network architectures in Appendix D.1-D. for all benchmarks.
Revised: We have provided a detailed discussion about training hyperparameters, experimental setup, and modiﬁcations made in publicly available network architectures in Appendix D.1-D. for all benchmarks.

Original:  We have added a comment directed to thereviewers and area chairs to provide a link to our training and inference code with trained models.We will publicly release our code if accepted. Code
Revised: Code is available at https://github.com/ubc-aamodt-group/BEL_ regression . We have provided the training and inference code with trained models.

Original:  of Ethics Some of the major applications of regression problems are artiﬁcial intelligence and autonomous machines, and regression improvement can accelerate the development of autonomous systems.
Revised: Code of Ethics Some of the major applications of regression problems are artiﬁcial intelligence and autonomous machines and regression improvement can accelerate the development of autonomous systems.

Original: However, depending upon the use, autonomous systems can have some negative societal impacts, such as job loss in some sectors and ethical concerns
Revised: However, depending upon the use, autonomous systems can have some negative societal impacts, such as job loss in some sectors and ethical concerns

Original: A A BLATION STUDY
Revised: A A BLATION STUDY

Original: Impact of BEL encoding, decoding, and loss functions:
Revised: Impact of combination of encoding, decoding, and loss functions:

Original: We propose multiple combinations of encoding, decoding [Table] and loss functions that can be used with BEL.
Revised: We propose multiple combinations of encoding, decoding, and loss functions that can be used with BEL.

Original: In Tables 3- 13, we show the effect of each combination of encoding, decoding, and loss function on the error of the model.
Revised: In Tables 3- 13, we show the effect of each combination of encoding, decoding, and loss function on the error of the model.

Original: Although general trends exist and some combinations perform consistently well across dataset, the optimal combination varies based on the dataset.
Revised: Although general trends exist and some combinations perform consistently well across datasets, the optimal combination varies based on the dataset

Original: [Table]
Revised: [Table]

Original: [Table] [Table] 
Revised: [Table] [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: -W18 feature extractor (FLD2)
Revised: W18 feature extractor (FLD2)

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Impact of quantization and decoding functions:
Revised: Impact of quantization and decoding functions:

Original: As discussed in Section 3, a real-valued label is quantized to a discrete value in { 1 , 2 , ..., N } .
Revised: As discussed in Section 3, a real-valued label is quantized to a discrete value in { 1 , 2 , ..., N } before applying the encoding function. In

Original: In Table 14, we show the effect of increasing the number of quantization levels ( N ) on the error for correlation-based decoding ( D GEN , which returns a quantized prediction) and expected correlation-based decoding ( D GEN-EX , which returns a continuous prediction).
Revised: Table 14, we show the effect of increasing the number of quantization levels ( N ) on the error for correlation-based decoding ( D GEN , which returns a quantized prediction) and expected correlationbased decoding ( D GEN-EX , which returns a continuous prediction).

Original: We vary the number of quantization levels from 64 to 256 and observe that there exists a tradeoff between reducing quantization error and using fewer classiﬁers.
Revised: As shown in the table, there exists a tradeoff between reducing quantization error and using fewer classiﬁers.

Original: The error is lower forquantization levels than it is for 256 as the improvement resulting from fewer binary classiﬁers is higher than the increase in quantization error.
Revised: The error is lower for 128 quantization levels than it is for 256 as the improvement resulting from fewer binary classiﬁers is higher than the increase in quantization error.

Revised: Moreover, the use of proposed decoding function

Original: Moreover, the use of D GEN-EX  consistently results in lower error compared to D GEN .
Revised: D GEN-EX for regression consistently results in lower error compared to D GEN .

Original: [Table]
Revised: [Table]

Original: Effect of the number of training samples on BEL: As discussed in Section 4, the performance of different encoding functions varies depending on the availability of sufﬁcient training data.
Revised: Impact of the number of training samples on BEL: As discussed in Section 4, the performance of different encoding functions varies depending on the availability of sufﬁcient training data.

Original: In
Revised: In

Original: Table 15, we analyze the effect of the number of available training samples for both simple and complex encodings.
Revised: Table 15, we analyze the effect of the number of available training samples for both simple and complex encodings.

Revised: We use the number of bit transitions as a measure of the complexity of a classiﬁer.

Original: As the number of training samples decreases, simpler encodings (U and J) perform better than more complex encodings (B1JDJ, B2JDJ, and Hex).
Revised: As the number of training samples decreases, simpler encodings (U and J) perform better than more complex encodings (B1JDJ, B2JDJ, and HEXJ).

Original: Using a more complex encoding reduces the number of classiﬁers; however, it increases each classiﬁer’s complexity
Revised: Using a more complex encoding reduces the number of classiﬁers; however, it increases each classiﬁer’s complexity

Original: (i
Revised: (i

Original: [Table]
Revised: [Table]

Original: e. the number of bit transitions) and thus performs poorly with less training data.
Revised: e. the number of bit transitions) and thus performs poorly with less training data.

Original: [Table]
Revised: [Table]

Original: Effect of reﬂected binary conversion:
Revised: Impact of reﬂected binary conversion:

Original: As mentioned in Section 3.2, we use reﬂected binary to increase the distance between distant labels based on the design properties of suitable regression encodings we proposed.
Revised: As mentioned in Section 3. [Table] we use reﬂected binary to increase the distance between distant labels based on the design properties of suitable regression encodings we proposed.

Original: Table 16 shows the impact of using reﬂected binary conversion on error for facial landmark detection benchmarks.
Revised: Table 16 shows the impact of using reﬂected binary conversion on error for facial landmark detection benchmarks.

Original: As shown in the table, the use of reﬂected binary signiﬁcantly reduces the error.
Revised: As shown in the table, the use of reﬂected binary signiﬁcantly reduces the error.

Original: [Table]
Revised: [Table]

Original: Use of binary heamaps:
Revised: Use of binary heamaps:

Original: Facial landmark detection approaches typically use heatmap regression.
Revised: Facial landmark detection approaches typically use heatmap regression.

Original: We also evaluate BEL-H-x, in which the real-valued heatmaps are converted to binary heatmaps with 8 quantization levels.
Revised: We also evaluate BEL-H-x, in which the real-valued heatmaps are converted to binary heatmaps with 8 quantization levels.

Original: Table 17 shows the impact of using binary heatmaps on error for facial landmark detection benchmarks.
Revised: Table 17 shows the impact of using binary heatmaps on error for facial landmark detection benchmarks.

Original: In BEL-H-x, each real-valued pixel of the target heatmap is converted to a binary code using eight quantization levels.

Original: For unary code, a 64 × 64 real-valued heatmap of one facial landmark is converted to eight 64x64 binary heatmaps, resulting in 32 , 768 ( 8 × 64 × 64 ) binary classiﬁers compared to 512 for BEL-U.
Revised: For unary code, a 64 × 64 real-valued heatmap of one facial landmark is converted to eight 64 × 64 binary heatmaps, resulting in 32 , 768 ( 8 × 64 × 64 ) binary classiﬁers compared to 512 for BEL-U.

Original: We believe that training a high number of binary classiﬁers results in high error for BEL-H-x.
Revised: We believe that training a high number of binary classiﬁers results in high error for BEL-H-x.

Original: [Table]
Revised: [Table]

Original: Hyperparameter θ : As shown in Figure 4b, we introduce a feature vector of size θ before the output layer.
Revised: Hyperparameter θ : As shown in Figure 4b, we introduce a feature vector of size θ before the output layer.

Original: Figure 6 compares the decrease in the error for different encodings and θ values.
Revised: Figure 6 compares the decrease in the error for different encodings and θ values.

Original: We observe that more complex encodingsthat show lower correlation between the classiﬁers’ outputs beneﬁt more from an increase in the value of θ , while a lower value of θ can be used for encodings with a high correlation between the classiﬁers’ outputs.
Revised: We observe that more complex encodings beneﬁt more from an increase in the value of θ , while a lower value of θ can be used for simpler encodings.

Original: [Figure]
Revised: [Figure]

Revised: l u t e i m p

Revised: r

Revised: o v e

Revised: m e n t i n N

Original: Effect of increasing the number of fully connected layers: For BEL, we propose to add a fully connected  layer in the regressor to reduce the feature vector size to θ and thus decrease the number of parameters in the regressor.
Revised: M E J U [Figure] Impact of increasing the number of fully connected layers: For BEL, we propose to add a fully connected bottleneck layer in the regressor to reduce the feature vector size to θ and thus decrease the number of parameters in the regressor.

Original: We perform an ablation study to study the impact of these added fully connected layers on  direct regression, multiclass classiﬁcation, and binary encoded labels.
Revised: We perform an ablation study to study the impact of this added fully connected layer on relative performance of direct regression, multiclass classiﬁcation, and binary encoded labels.

Original: Tableprovides the error (MAE or NME) for direct regression and multiclass classiﬁcation with one or two fully connected layers after the feature extractor.
Revised: Table 18 provides the error (MAE or NME) for direct regression and multiclass classiﬁcation with one or two fully connected layers after the feature extractor.

Original: Table 19 shows the impact of increasing the number of fully connected layers up to three on the error (MAE or NME) for different regression approaches.
Revised: Further, we evaluate BEL, direct regression, and multiclass classiﬁcation for higher number of fully connected layers as shown in Table 19.

Original: We observe that increasing the number of fully connected layers in direct regression and multiclass classiﬁcation does not improve the accuracy for most benchmarks (possibly due to overparameterization).
Revised: We observe that increasing the number of fully connected layers in direct regression and multiclass classiﬁcation does not improve the accuracy for most benchmarks (possibly due to overparameterization).

Original: BEL with two fully connected layers outperforms direct regression and multiclass classiﬁcation in both cases.
Revised: BEL with two fully connected layers outperforms direct regression and multiclass classiﬁcation in both cases.

Original: Furthermore, even for a higher number of fully connected  layers in BEL, different encoding functions result in the lowest error.
Revised: Furthermore, even for a higher number of fully connected [Table] layers in BEL, the suitability of an encoding function varies with the dataset, demonstrating the importance of BEL design space

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: Training-validation set based evaluation Ideally, a validation set should be used for model selection.
Revised: Training-validation set based evaluation: Ideally, a validation set should be used for model selection.

Original: Hence we have reevaluated the benchmarks with20% of the training data being used as a validation set to select the best  model (i.e., which model is the best over multiple epochs).
Revised: Hence we have reevaluated the benchmarks with a validation set to select the best design parameters and the best model (i.e., which model is the best over multiple epochs).

Original: Datasets used in benchmarks do not provide separate validation datasets. We use 80% of the training data for training and use validation error to select the best design parameters for BEL and the best model for all approaches.
Revised: Since datasets used in benchmarks do not provide separate validation datasets, we use 20% of the training data as a validation set.

Original: Since earlier works use 100% training data for the reported results and use test error for model selection, we have re-run specialized approaches (if possible), direct regression, and multiclass classiﬁcation.
Revised: Since earlier works use 100% training data for the reported results and use test error for model selection, we have re-run specialized approaches (if possible), direct regression, and multiclass classiﬁcation.

Original: It was not possible for us to re-run experiments for all specialized approaches due to resource constraints, and the comparison is conservative for many benchmarks since we use less data while selecting the best BEL approach than prior works used for model selection.
Revised: It was not possible for us to re-run experiments for all specialized approaches due to resource constraints, and the comparison is conservative for many benchmarks.

Original: Table 20 compares different regression approaches for this evaluation setup.
Revised: Table 20 compares different regression approaches for this evaluation setup.

Original: Note that the additional results do not diminish the effectiveness of BEL and BEL outperforms direct regression and multiclass classiﬁcation for all benchmarks and specialized approaches for several benchmarks.
Revised: Note that the additional results do not diminish the effectiveness of BEL and BEL outperforms direct regression and multiclass classiﬁcation for all benchmarks and specialized approaches for several benchmarks.

Original: Here, r is a scaling factor for the error probabilities.

Original: B E XPECTED E RROR D ERIVATION
Revised: B E XPECTED E RROR D ERIVATION

Original: This section explains the expected error equations used to compare BEL-U and BEL-J in Section 3.1.
Revised: This section explains the expected error equations used to compare BEL-U and BEL-J in Section 3.1.

Original: We ﬁrst explain the  decoding function used for BEL-U and derive the relation between the expected  errorof labels and binary classiﬁers for BEL-U.
Revised: We ﬁrst explain the encoding and decoding function used for BEL-U and derive the relation between the expected regression error and classiﬁcation error for BEL-U.

Original: Then, we explain the encoding/decoding functions and expected error relation for BEL-J.
Revised: Then, we explain the encoding/decoding functions and expected error relation for BEL-J

Original:  B. P RELIMINARIES
Revised: . B. P RELIMINARIES

Original: We compose binary-encoded labels as a quantization function q , an encoding function E , and a decoding function D .

Original: Given a sample i drawn from a dataset with minimum label a and maximum label b , let y i ∈
Revised: Given a sample i drawn from a dataset with minimum label a and maximum label b , let y i ∈

Original: [ a, b ] represent the target label for that sample.
Revised: [ a, b ] represent the target label for that sample.

Original: Assuming uniform quantization, the range of target labels can be quantized using q : [ a, b ] → { 1 , 2 , ..., N− 1 } through equation 7.
Revised: Assuming uniform quantization, the range of target labels can be quantized using q : [ a, b ] → { 1 , 2 , ..., N } through Equation 7.

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: We deﬁne the encoding function E : { 1 , 2 , ..., N − 1 } → { 0 , 1 } M to convert a target quantized level
Revised: We deﬁne the encoding function E : { 1 , 2 , ..., N − 1 } → { 0 , 1 } M to convert a target quantized level

Original: Q i ∈ { 1 , 2 , ..., N − 1 } to a binary code B i ∈ { 0 , 1 } M .
Revised: Q i ∈ { 1 , 2 , ..., N − 1 } to a binary code B i ∈ { 0 , 1 } M .

Original: We further deﬁne the decoding function
Revised: We further deﬁne the decoding function

Original: D : { 0 , 1 } M → [ a, b ] to convert the predicted binary code ˆ B i to the predicted label ˆ y i .
Revised: D : { 0 , 1 } M → [ a, b ] to convert the predicted binary code ˆ B i to the predicted label ˆ y i .

Original: Although the decoding functions used in this analysis predict the quantized label and introduce quantization error, we do not include quantization error in the expected label error for our analysis and it is constant for both BEL-U and BEL-J. The expected value of absolute error between the target y and predicted labels ˆ y is used for the analysis as typically mean absolute error is used as the evaluation metric in regression problems.
Revised: Although the decoding functions used in this analysis predict the quantized label and introduce quantization error [Table] we do not include quantization error in the expected absolute error for our analysis as it is constant for both BEL-U and BEL-J. The expected value of absolute error between the target y and predicted labels ˆ y is used for the analysis as typically mean absolute error is used as the evaluation metric in regression problems.

Original: Let us denote the error probability of a binary classiﬁer C k used to predict bit k in a binary code
Revised: Let us denote the error probability of a binary classiﬁer C k used to predict bit k in a binary code

Original: B i
Revised: B i

Original: = E ( n ) as e k ( n ) , where n is the target quantized label Q i .
Revised: = E ( n ) as e k ( n ) , where n is the target quantized label Q i .

Original: Then, e
Revised: Then, e

Original: [Equation]
Revised: [Equation]

Original: = Pr (ˆ b ki = F ) for target labelQ i = n and target binary code B i
Revised: target label

Original: [Equation]
Revised: [Equation]

Original: E
Revised: and target binary code

Original: [Equation]
Revised: [Equation]

Original: (8) where ˆ b ki = T indicate a correct binary classiﬁcation by classiﬁer C k ( ˆ b ki ==
Revised: ) where ˆ b ki = T indicates a correct binary classiﬁcation by classiﬁer C k ( ˆ b ki = =

Original: b ki ) for sample
Revised: b ki ) for sample

Original: i and ˆ b ki = F indicat an incorrect binary classiﬁcation by classiﬁer C k ( ˆ b ki (cid:54) =
Revised: i and ˆ b ki = F indicates an incorrect binary classiﬁcation by classiﬁer C k ( ˆ b ki (cid:54) =

Original: b ki ) for sample i.
Revised: b ki ) for sample i .

Original: B. E XPECTED E RROR FOR BEL-U
Revised: B. E XPECTED E RROR FOR BEL-U

Original: Encoding and decoding functions The encoding and decoding functions for BEL-U are deﬁned as:
Revised: Encoding and decoding functions: The encoding and decoding functions for BEL-U are deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: , where b ki = (cid:26) 1  k < Q i [-
-] Otherwise
Revised: , where b ki = (cid:26) 1 , k < Q i [-
-]0 , Otherwise

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Expected error For target quantized label Q i = n ( n ∈ { 1 , 2 , ..., N − 1 } ), ignoring the quantization error, the expected error between target y i and predicted label ˆ y i can be derived as:
Revised: Expected error: For target quantized label Q i = n ( n ∈ { 1 , 2 , ..., N − 1 } ), ignoring the quantization error, the expected error between target y i and predicted label ˆ y i can be derived as:

Original: [Equation]
Revised: [Equation]

Original: equation 8)

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: For a uniform distribution of target labels in the range [1 , N − 1] , the expected error can be derived as:
Revised: For a uniform distribution of target labels in the range [1 , N − 1] , the expected error can be derived as:

Original: [Equation]
Revised: [Equation]

Original: B. E XPECTED E RROR FOR BEL-J
Revised: B. E XPECTED E RROR FOR BEL-J

Original: Encoding and decoding functions For target quantized label Q i ∈ { 1 , 2 , ..., N − 1 } , BEL-J encoding requires N 2 bits/binary classiﬁers.
Revised: Encoding and decoding functions: For target quantized label Q i ∈ { 1 , 2 , ..., N − 1 } , BEL-J encoding requires N 2 bits/binary classiﬁers.

Original: The encoding for BEL-J can be deﬁned as:
Revised: The encoding for BEL-J can be deﬁned as:

Original: [Equation]
Revised: [Equation]

Original:  2 i , .., b N 2{+
+} i ,where b ki = (cid:26) 
Revised: J ( Q i ) = b 1 i , b 2 i , .., b N {+
+} i ,where b ki = (cid:26) 1 ,

Original: N 2 − Q
Revised: N 2 − Q

Original: i < k (cid:54) N − Q i [-
-] Otherwise (13)
Revised: i < k (cid:54) N − Q i [-
-]0 , Otherwise (13)

Original: B i
Revised: B i

Original: C 1 C 2 C 3 C
Revised: C 1 C 2 C 3 C

Original: Q i - - - - - - - + + + + + + + + + + + + + +
Revised: Q i - - - - - - - + + + + + + + + + + + + + +

Original: Tl Tf Tc
Revised: Tl Tf Tc

Original: Decodingfunction D
Revised: Decodingfunction D

Original: Decodedoutput
Revised: Decodedoutput

Original: Tl + Tf + Tc
Revised: Tl + Tf + Tc

Original: N2-n+  

Revised: N2-n+

Original: Bit position 1 B
Revised: Bit position 1 B

Original: i
Revised: i

Original: N2-n+  .....
Revised: N2-n+  .....

Original: ..... N2-n
Revised: ..... N2-n

Original: N2-1 N2 .....
Revised: N2-1 N2 .....

Original: N2-nk .....
Revised: N2-nk .....

Original: .....k-
Revised: .....k-

Original:  - - - - Case 1: B i - ..... ..... - 1 .....|
Revised:  - - - - Case 1: B i - ..... ..... - 1 .....|

Original: Tf i -
Revised: Tf i -

Original: Tf i | =
Revised: Tf i | =

Original: ( N / 2)- n +1- k - -0 Case 2: B i ..... .....
Revised: ( N / 2)- n +1- k - -0 Case 2: B i ..... .....

Original: .....|
Revised: .....|

Original: Tf i - Tf i | =
Revised: Tf i - Tf i | =

Original: N2-n+  - 1 ^ ^ ^ ^
Revised: N2-n+  - 1 ^ ^ ^ ^

Original: Similarly, the decoding functions for BEL-J can be deﬁned as:
Revised: Similarly, the decoding functions for BEL-J can be deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where, Tl (
Revised: where, Tl (

Original: [Equation]
Revised: [Equation]

Original: In equation 14, Tl () ﬁnds the location of the last occurrence of “ 1 ” in the predicted binary code ˆ B i .
Revised: In Equation 14, Tl () ﬁnds the location of the last occurrence of “ 1 ” in the predicted binary code ˆ B i .

Original: Similarly, Tf () ﬁnds the location of the ﬁrst occurrence of “ 1 ” in the binary code ˆ B i .
Revised: Similarly, Tf () ﬁnds the location of the ﬁrst occurrence of “ 1 ” in the binary code ˆ B i .

Original: Figure 7 gives examples of binary codes for label Q i ∈ { 1 , 2 , ..., 7 } and the corresponding values of the different terms in equation 14.
Revised: Figure 7 gives examples of binary codes for label Q i ∈ { 1 , 2 , ..., 7 } and the corresponding values of the different terms in Equation 14.

Original: For example, for label Q i = 3 , the binary code is “ 0111 ”.
Revised: For example, for label Q i = 3 , the binary code is “ 0111 ”.

Original: Here, the last occurrence of “ 1 ” is at position 4 , and Tl = − 4 .
Revised: Here, the last occurrence of “ 1 ” is at position 4 , and Tl = − 4 .

Original: Similarly, the ﬁrst occurrence of “ 1 ” is at position 2 , and Tf = (4 + 1) − 2 = 3 .
Revised: Similarly, the ﬁrst occurrence of “ 1 ” is at position 2 , and Tf = (4 + 1) − 2 = 3 .

Original: Expected error For BEL-J code, binary classiﬁers ( C 1 , C 2 , ..., C N 2 ) are used.
Revised: Expected error: For BEL-J code, binary classiﬁers ( C 1 , C 2 , ..., C N 2 ) are used.

Original: For a given input sample i, an error in any of the binary classiﬁers’ outputs (ˆ b 1 i , ˆ b 2 i , ..., ˆ b N 2 ) will result in an error between Tf ( ˆ B i ) /Tl
Revised: For a given input sample i, an error in any of the binary classiﬁers’ outputs (ˆ b 1 i , ˆ b 2 i , ..., ˆ b N 2 ) will result in an error between Tf ( ˆ B i ) /Tl

Original: ( ˆ B i ) and Tf ( B i ) /Tl
Revised: ( ˆ B i ) and Tf ( B i ) /Tl

Original: ( B i ) in equation 14.
Revised: ( B i ) in Equation 14.

Original: We refer to Tf ( ˆ B i ) and Tl ( ˆ B i ) as ˆ Tf i and ˆ Tl
Revised: We refer to Tf ( ˆ B i ) and Tl ( ˆ B i ) as ˆ Tf i and ˆ Tl

Original: i (predicted binary code), and Tf ( B i ) and Tl ( B i ) as Tf i and Tl i (target binary code) for brevity.
Revised: i (predicted binary code), and Tf ( B i ) and Tl ( B i ) as Tf i and Tl i (target binary code) for brevity.

Original: Expected value of the absolute error can be further expanded as:
Revised: Expected value of the absolute error can be further expanded as:

Original: [Equation]
Revised: [Equation]

Original: Thus, the sum of the expected error of Tf () and Tl () is the upper bound of the label’s expected error.
Revised: Thus, the sum of the expected error of Tf () and Tl () is the upper bound of the label’s expected error.

Original: Further, we derive the relation between binary classiﬁers’ error probabilities and E ( | ˆ Tf i − Tf i | ) and
Revised: Further, we derive the relation between binary classiﬁers’ error probabilities and E ( | ˆ Tf i − Tf i | ) and

Original: E ( | ˆ Tl i − Tl i | ) .
Revised: E ( | ˆ Tl i − Tl i | ) .

Original: We consider Q i = n, where 1 (cid:54) n (cid:54) N 2 for our derivation.
Revised: We consider Q i = n, where 1 (cid:54) n (cid:54) N 2 for our derivation.

Original: In such a case, Tf i = n and Tl i = − N 2 .
Revised: In such a case, Tf i = n and Tl i = − N 2 .

Original: However, as the code is symmetric around Q i = N 2 , it can be shown that the derived equation for E | ˆ y i − y i | can be used for 1 (cid:54) Q i (cid:54) N − 1 .
Revised: However, as the code is symmetric around Q i = N 2 , it can be shown that the derived equation for E | ˆ y i − y i | can be used for 1 (cid:54) Q i (cid:54) N − 1 .

Original: Derivation of E | ˆ
Revised: Derivation of E | ˆ

Original: Tf i − Tf i | : As shown in equation 14, Tf () ﬁnds the location k of the ﬁrst occurrence of “ 1 ” in the binary sequence.
Revised: Tf i − Tf i | : As shown in Equation 14, Tf () ﬁnds the location k of the ﬁrst occurrence of “ 1 ” in the binary sequence.

Original: In the case of an erroneous binary sequence, the position of the ﬁrst occurrence of “ 1 ” might shift, which results in an error between ˆ Tf i and Tf i .
Revised: In the case of an erroneous binary sequence, the position of the ﬁrst occurrence of “ 1 ” might shift, which results in an error between ˆ Tf i and Tf i .

Original: Figure 8 shows examples of the correct and erroneous outputs of classiﬁers for label Q i = n .
Revised: Figure 8 shows examples of the correct and erroneous outputs of classiﬁers for label Q i = n .

Original: For label Q i = n , b k i = 0 for k ∈ { 1 , 2 , ..., N 2 − n } and b k i = 1 for k ∈ { N 2 − n + 1 , N 2 − n + 2 , ..., N 2 } .
Revised: For label Q i = n , b k i = 0 for k ∈ { 1 , 2 , ..., N 2 − n } and b k i = 1 for k ∈ { N 2 − n + 1 , N 2 − n + 2 , ..., N 2 } .

Original: For case 1, error in a classiﬁer C k , k ∈ { 1 , 2 , ..., N 2 − n } is considered, where b ki = 0 and ˆ b ki = 1 .
Revised: For case 1, error in a classiﬁer C k , k ∈ { 1 , 2 , ..., N 2 − n } is considered, where b ki = 0 and ˆ b ki = 1 .

Original: For k ∈ { 1 , 2 , ..., N 2 − n } , an error at classiﬁer C k will result in erroneous ˆ Tf i only if all proceeding classiﬁers are correct, since if any of the proceeding classiﬁer z is incorrect, i.e. ˆ b zi = 1 , then the location of the ﬁrst occurrence of “ 1 ” will be shifted to z , and any error in the following classiﬁers will not affect the value of ˆ Tf .
Revised: For k ∈ { 1 , 2 , ..., N 2 − n } , an error at classiﬁer C k will result in erroneous ˆ Tf i only if all proceeding classiﬁers are correct, since if any of the proceeding classiﬁer z is incorrect, i.e. ˆ b zi = 1 , then the location of the ﬁrst occurrence of “ 1 ” will be shifted to z , and any error in the following classiﬁers will not affect the value of ˆ Tf .

Original: Such a case ( ˆ b 1 i =
Revised: Such a case ( ˆ b 1 i =

Original: T, ˆ b 2 i
Revised: T, ˆ b 2 i

Original: = T, ..., ˆ b k − 1 i
Revised: = T, ..., ˆ b k − 1 i

Original: = T, ˆ b ki 
Revised: = T, ˆ b ki = F,

Original: [Equation]
Revised: [Equation]

Original: F,
Revised: k

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: T/F, ..., ˆ b
Revised: T/F, ..., ˆ b

Original: N 

Revised: N

Original: i
Revised: i

Original: =
Revised: =

Original: T/F ) considers a total of 2 N 2 − k combinations out of 2 N 2 for k ∈ { 1 , 2 , ..., N 2 − n } .
Revised: T/F ) considers a total of 2 N 2 − k combinations out of 2 N 2 for k ∈ { 1 , 2 , ..., N 2 − n } .

Original: Assuming that the binary classiﬁers are mutually independent, the error value and the probability of this combination can be shown to be:
Revised: Assuming that the binary classiﬁers are mutually independent, the error value and the probability of this combination can be shown to be:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: The above term considers combinations ( b (cid:48) 1 = T, b
Revised: The above term considers combinations ( b (cid:48) 1 = T, b

Original: [Equation]
Revised: [Equation]

Original: 2 =T, ..., b [Equation] k
Revised: 2 =

Original: [Equation]
Revised: [Equation]

Original: = T, b
Revised: , ..., b

Original: [Equation]
Revised: [Equation]

Original: = F, b
Revised: b

Original: [Equation]
Revised: [Equation]

Original: k +
Revised: b

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: T/F, ..., b (cid:48)
Revised: T/F, ..., b (cid:48)

Original: N 2 = T/F ) for k ∈ { 1 , 2 , ..., N 2 − n } , which constitutes to a total of (cid:80) N 2 − n k
Revised: N 2 = T/F ) for k ∈ { 1 , 2 , ..., N 2 − n } , which constitutes to a total of (cid:80) N 2 − n k

Original: =1 2 N 2 − k combinations out of 2 N 2 .
Revised: =1 2 N 2 − k combinations out of 2 N 2 .

Original: For case 2, error in a classiﬁer C k , k ∈ { N 2 − n + 1 , N 2 − n + 2 , ..., N 2 } is considered, where b ki =and ˆ b ki = 0 .
Revised: For case 2, error in a classiﬁer C k , k ∈ { N 2 − n + 1 , N 2 − n + 2 , ..., N 2 } is considered, where b ki =and ˆ b ki = 0 .

Original: We consider a combination ( ˆ b 1 i
Revised: We consider a combination ( ˆ b 1 i

Original: = T, ˆ b 2 i
Revised: = T, ˆ b 2 i

Original: = T, ..., ˆ 
Revised: = T, ..., ˆ b N

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Revised: =

Revised: [Equation]

Revised: [Equation]

Original: F, ˆ b ki = T, ˆ b k
Revised: F, ˆ b ki = T, ˆ b k

Original: +1 i
Revised: +1 i

Original: = T/F, ..., ˆ [Equation] N
Revised: = T/F, ..., ˆ b N

Original: [Equation]
Revised: [Equation]

Original: T/
Revised: T/

Original: [Equation]
Revised: [Equation]

Original: i ).
Revised: i ).

Original: For this case, the position of the ﬁrst occurrence of “ 1 ” will be moved to k , which will result in erroneous
Revised: For this case, the position of the ﬁrst occurrence of “ 1 ” will be moved to k , which will result in erroneous

Original: ˆ Tf
Revised: ˆ Tf

Original: i .
Revised: i .

Original: Such a case would cover 2 N 2 − k combinations out of 2 N 2 for k ∈ { N 2 − n + 1 , N 2 − n + 2 , ..., N 2 } .
Revised: Such a case would cover 2 N 2 − k combinations out of 2 N 2 for k ∈ { N 2 − n + 1 , N 2 − n + 2 , ..., N 2 } .

Original: The error value and the probability of this combination can be shown to be:
Revised: The error value and the probability of this combination can be shown to be:

Original: [Equation]
Revised: [Equation]

Original: The above term considers combinations (ˆ b 1 i
Revised: The above term considers combinations (ˆ b 1 i

Original: =
Revised: =

Original: T, ˆ b 2 i
Revised: T, ˆ b 2 i

Original: = T, ..., ˆ b N 2 − n i
Revised: = T, ..., ˆ b N 2 − n i

Original: [Equation]
Revised: [Equation]

Original: =F [Equation]
Revised: =

Original:  ..., ˆ b k − 1 i =
Revised: F, ..., ˆ b k − 1 i =

Original: F, ˆ b ki = T, ˆ b k
Revised: F, ˆ b ki = T, ˆ b k

Original: +1 i
Revised: +1 i

Original: = T/F, ..., ˆ b N 2 = T/F i ) , which constitutes to a total of (cid:80) N 2 k
Revised: = T/F, ..., ˆ b N 2 = T/F i ) , which constitutes to a total of (cid:80) N 2 k

Original: = N 2 − n +1 2 N 2 − k combinations out of 2 N 2 for k ∈ { N 2 − n + 1 , N 2 − n + 2 , ..., N 2 } .
Revised: = N 2 − n +1 2 N 2 − k combinations out of 2 N 2 for k ∈ { N 2 − n + 1 , N 2 − n + 2 , ..., N 2 } .

Original: Combining equation 16-equation 19, the expected value of | ˆ Tf i − Tf i | can be derived as:E
Revised: Combining Equation 16 to Equation 19, the expected value of | ˆ Tf i − Tf i | can be derived as:

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst term in equation 20 covers (cid:80) N 2 − n k
Revised: The ﬁrst term in Equation 20 covers (cid:80) N 2 − n k

Original: =1 2 N 2 − k combinations and the second term considers (cid:80) N 2 k
Revised: =1 2 N 2 − k combinations and the second term considers (cid:80) N 2 k

Original: = N 2 − n +1 2 N 2 − k combinations.
Revised: = N 2 − n +1 2 N 2 − k combinations.

Original: Adding one combination where all the classiﬁers are correct, equation 20 considers all of the possible combinations 2 N 2 to ﬁnd expected value of | ˆ Tf i − Tf i | .
Revised: Adding one combination where all the classiﬁers are correct, Equation 20 considers all of the possible combinations 2 N 2 to ﬁnd expected value of | ˆ Tf i − Tf i | .

Original: N2-n+  1 

Revised: N2-n+  1

Original: N2-n+ 1 .....
Revised: N2-n+ 1 .....

Original: ..... N2-n
Revised: ..... N2-n

Original: N2-1 N2 .....
Revised: N2-1 N2 .....

Original: N2-nk .....
Revised: N2-nk .....

Original: .....k- - 0 0 - - - ..... ..... - - ..... - |
Revised: .....k- - 0 0 - - - ..... ..... - - ..... - |

Original: Tl i - Tl
Revised: Tl i - Tl

Original: i | =
Revised: i | =

Original: ( N / 2)- k +  0 0 

Revised: ( N / 2)- k +  0 0

Original: | Tl i - Tl i | =
Revised: | Tl i - Tl i | =

Original: N2Bit position
Revised: N2Bit position

Original: B i
Revised: B i

Original: Case 1: B i
Revised: Case 1: B i

Original: Case 2: B i ^ ^
Revised: Case 2: B i ^ ^

Original: ^ ^
Revised: ^ ^

Original: Derivation of E | ˆ Tl i − Tl i | : As shown in equation 14, Tl () ﬁnds the location k of the last occurrence of “ 1 ” in the binary sequence.
Revised: Derivation of E | ˆ Tl i − Tl i | : As shown in Equation 14, Tl () ﬁnds the location k of the last occurrence of “ 1 ” in the binary sequence.

Original: In the case of an erroneous binary sequence, the position of the last occurrence of “ 1 ” might shift, which results in an erroneous value of ˆ Tl i .
Revised: In the case of an erroneous binary sequence, the position of the last occurrence of “ 1 ” might shift, which results in an erroneous value of ˆ Tl i .

Original: Figure 9 shows examples of correct and erroneous outputs of classiﬁers for label Q i = n .
Revised: Figure 9 shows examples of correct and erroneous outputs of classiﬁers for label Q i = n .

Original: For case 1, an error in a classiﬁer C k , k ∈ { N 2 − n + 1 , N 2 − n + 2 , ..., N 2 } is considered, where b ki = 1 and ˆ b ki = 0 .
Revised: For case 1, an error in a classiﬁer C k , k ∈ { N 2 − n + 1 , N 2 − n + 2 , ..., N 2 } is considered, where b ki = 1 and ˆ b ki = 0 .

Original: We consider a combination (ˆ b N 2 i =
Revised: We consider a combination (ˆ b N 2 i =

Original: F, ˆ b N 2 − 1 i = F, ..., ˆ b k
Revised: F, ˆ b N 2 − 1 i = F, ..., ˆ b k

Original: +1 i
Revised: +1 i

Original: = F, ˆ b ki = T, ˆ b k − 1 i =
Revised: = F, ˆ b ki = T, ˆ b k − 1 i =

Original: T/F, ..., ˆ b 

Revised: T/F, ..., ˆ b

Original: i = T/F ) .
Revised: i = T/F ) .

Original: For this case, position of the last occurrence of “ 1 ” will be moved to k , which will result in erroneous ˆ Tl i .
Revised: For this case, position of the last occurrence of “ 1 ” will be moved to k , which will result in erroneous ˆ Tl i .

Original: Such a case would cover 2 k − 1 combinations out of 2 N 2 .
Revised: Such a case would cover 2 k − 1 combinations out of 2 N 2 .

Original: The error value and the probability of this combination can be shown to be:
Revised: The error value and the probability of this combination can be shown to be:

Original: [Equation]
Revised: [Equation]

Original: The above term considers combinations (ˆ b N 

Revised: The above term considers combinations (ˆ b N

Original: i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: T/F, ..., ˆ b 

Revised: T/F, ..., ˆ b

Original: i = T/F ) for k ∈ { N 2 − n + 1 , N 2 − n + 2 , ..., N 2 } , which constitutes to a total of (cid:80) N 2 k = N 2 − n +1 2 k − 1 combinations out of 2 N 2 .
Revised: i = T/F ) for k ∈ { N 2 − n + 1 , N 2 − n + 2 , ..., N 2 } , which constitutes to a total of (cid:80) N 2 k = N 2 − n +1 2 k − 1 combinations out of 2 N 2 .

Original: For case 2, an error in a classiﬁer C k , k ∈ { 1 , 2 , ..., N 2 − n } is considered, where b ki = 0 and ˆ b ki = 1 .
Revised: For case 2, an error in a classiﬁer C k , k ∈ { 1 , 2 , ..., N 2 − n } is considered, where b ki = 0 and ˆ b ki = 1 .

Original: We consider a combination ( ˆ b N 2 i = F, ..., ˆ b N 2 − n +1 i
Revised: We consider a combination ( ˆ b N 2 i = F, ..., ˆ b N 2 − n +1 i

Original: =
Revised: =

Original: F, ˆ b N 2 − n i
Revised: F, ˆ b N 2 − n i

Original: = T, ..., ˆ b k
Revised: [Equation] [Equation]

Original: [-+
 i-]
Revised: [-+
 i-]b k +

Original: [Equation]
Revised: [Equation]

Original: T, ˆ b ki = F, ˆ b k − 1 i =
Revised: ˆ b ki = F, ˆ b k − 1 i =

Original: T/F, ..., ˆ b 

Revised: T/F, ..., ˆ b

Original: i = T/F ).
Revised: i = T/F ).

Original: For this case, position of the last occurrence of “ 1 ” will be moved to k , which will result in erroneous ˆ Tl i .
Revised: For this case, position of the last occurrence of “ 1 ” will be moved to k , which will result in erroneous ˆ Tl i .

Original: Such a case would cover 2 k − 1 combinations out of 2 N 2 .
Revised: Such a case would cover 2 k − 1 combinations out of 2 N 2 .

Original: The error value and the probability of this combination can be shown to be:
Revised: The error value and the probability of this combination can be shown to be:

Original: [Equation]
Revised: [Equation]

Original: The above term considers combinations (ˆ b N 

Revised: The above term considers combinations (ˆ b N

Original: i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: = T, ˆ b ki = F, ˆ b k − 1 i = T/F, ..., ˆ b 1 i = T/F ) for k ∈ { 1 , 2 , ..., N 2 − n } , which constitutes to a total of (cid:80) N 2 − n k
Revised: T, ˆ b ki = F, ˆ b k − 1 i = T/F, ..., ˆ b 1 i = T/F ) for k ∈ { 1 , 2 , ..., N 2 − n } , which constitutes to a total of (cid:80) N 2 − n k

Original: =
Revised: =

Original: k − 1 combinations out of 2 N 2 .
Revised: k − 1 combinations out of 2 N 2 .

Original: Combining equation 21-equation 24, the expected value of | ˆ Tl i − Tl i | can be derived as:
Revised: Combining Equation 21 to Equation 24, the expected value of | ˆ Tl i − Tl i | can be derived as:

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst term in equation 25 covers (cid:80) N 2 k = N 2 − n +1 2 k − 1 combinations and the second term considers (cid:80) N 2 − n k
Revised: The ﬁrst term in Equation 25 covers (cid:80) N 2 k = N 2 − n +1 2 k − 1 combinations and the second term considers (cid:80) N 2 − n k

Original: =1 2 k − 1 combinations.
Revised: =1 2 k − 1 combinations.

Original: Adding one combination where all the classiﬁers are correct, equation 25 considers all of the possible combinations 2 N 2 to ﬁnd expected value of | ˆ Tl i − Tl i | .
Revised: Adding one combination where all the classiﬁers are correct, Equation 25 considers all of the possible combinations 2 N 2 to ﬁnd expected value of | ˆ Tl i − Tl i | .

Original: Combining equation 15, equation 20, and equation 25, the expected value ofaverage error  in terms of classiﬁers’ error probabilities can de derived as:
Revised: Combining Equation 15, Equation 20, and Equation 25, the expected value of error for Q i = n in terms of classiﬁers’ error probabilities can be derived as:

Original: [Equation]
Revised: [Equation]

Original: As the binary code is symmetric around N 2 as shown in Figure 7, the expected errors for label y i ∈ [1 , N 2 ] can be mirrored to ﬁnd expected errors for label y i ∈
Revised: As the binary code is symmetric around N 2 as shown in Figure 7, the expected errors for label y i ∈ [1 , N 2 ] can be mirrored to ﬁnd expected errors for label y i ∈

Original: [ N 2 , N − 1] .
Revised: [ N 2 , N − 1] .

Original: For a uniform distribution of target labels in the range [1 , N − 1] , the expected error can be derived as: 
Revised: For a uniform distribution of target labels in the range [1 , N − 1] , the expected error can be derived as: E

Original: [Equation]
Revised: [Equation]

Revised: BEL

Revised: [Equation]

Revised: [Equation]

Revised: n

Original:  We also verify the equation by comparing the expected value of error based on equation 26 for Q i ∈  1 , 2 , ..., N − 1  with the expected error calculated by 100 , 000 random samples of binary sequences for the same error probabilities e k ( n ) .
Revised: [Equation] N [Equation] We also verify the equation by comparing the expected value of error based on Equationfor Q i ∈ { 1 , 2 , ..., N − 1 } with the expected error calculated by 100 , 000 random samples of binary sequences for the same error probabilities e k ( n ) .

Original: Figure 10 compares the expected error from equation 26 and measured from statistical samples, and validates error upper bounds calculated using equationand equation 27.
Revised: Figure 10 compares the expected error from Equation 26 and measured from statistical samples, and validates error upper bounds calculated using Equation 26 and Equation 27.

Original: [Figure]
Revised: [Figure]

Original: C E RROR P ROBABILITY OF C LASSIFIERS
Revised: C E RROR P ROBABILITY OF C LASSIFIERS

Original: It is known that the error/misclassiﬁcation probability e k ( n ) of a classiﬁer tends to increase as the target label value n is closer to the classiﬁer’s decision boundaries Cardoso & Pinto da Costa (2007).
Revised: It is known that the error/misclassiﬁcation probability e k ( n ) of a classiﬁer tends to increase as the target label value n is closer to the classiﬁer’s decision boundaries (Cardoso & Pinto da Costa, 2007).

Original: We approximate e k ( y ) for a classiﬁer C k with t bit transitions as a linear combination of t
Revised: We approximate e k ( y ) for a classiﬁer C k with t bit transitions as a linear combination of t

Original: Gaussian distributions.
Revised: Gaussian distributions.

Original: Here, each Gaussian term is centered around a bit transition.
Revised: Here, each Gaussian term is centered around a bit transition.

Original: Figure 11 shows the empirically observed error probability distributions for different classiﬁers trained for different combinations of network and dataset.
Revised: Figure 11 shows the empirically observed error probability distributions for different classiﬁers trained for different combinations of network and dataset.

Original: We also show the approximate error probability distribution using a linear combination of Gaussian distributions.
Revised: We also show the approximate error probability distribution using a linear combination of Gaussian distributions.

Original: Here r is a scalar multiplied with probability density of gaussian distribution and σ2 is the variance (equation 3 and equation 4).
Revised: Here r is a scalar multiplied with probability density of gaussian distribution and σ is the standard deviation (Equation 3 and Equation 4).

Original: [Figure]
Revised: [Figure]

Original: D E XPERIMENTAL M ETHODOLOGY
Revised: D E XPERIMENTAL M ETHODOLOGY

Original: All experiments are conducted on a Linux machine with an Intel i9-9900X processor and a Nvidia
Revised: All experiments are conducted on a Linux machine with an Intel i9-9900X processor and an Nvidia

Original: RTX 2080 Ti GPU with 11GB of memory.
Revised: RTX 2080 Ti GPU with 11GB of memory.

Original: Our code is implemented using Python 3.8.3 with Pytorch 1.5.1 using CUDA 10.2.
Revised: Our code is implemented using Python 3.8.3 with Pytorch 1.5.1 using CUDA 10.2.

Original: Our evaluation is averaged over 5 training runs with separate seeds.
Revised: Our evaluation is averaged over 5 training runs with separate seeds.

Original: D. H EAD P OSE E STIMATION
Revised: D. H EAD P OSE E STIMATION

Original: Head pose estimation aims to ﬁnd
Revised: Head pose estimation aims to ﬁnd

Original: a human head’s pose in terms of three angles: yaw, pitch, and roll.
Revised: a human head’s pose in terms of three angles: yaw, pitch, and roll.

Original: In this work, we consider landmark-free 2D head pose estimation.
Revised: In this work, we consider landmark-free 2D head pose estimation.

Original: Datasets: We follow the evaluation setting of Hopenet (Ruiz et al., 2018) and FSA-Net (Yang et al., 2019) and use two evaluation protocols with three widely used datasets: 300W-LP (Zhu et al., 2016),
Revised: Datasets: We follow the evaluation setting of Hopenet (Ruiz et al., 2018) and FSA-Net (fsa, 2019) and use two evaluation protocols with three widely used datasets: 300W-LP (Zhu et al., 2016),

Original: BIWI (Fanelli et al., 2013), and AFLW2000 (Zhu et al., 2016).
Revised: BIWI (Fanelli et al., 2013), and AFLW2000 (Zhu et al., 2016).

Original: We use two evaluation protocols.
Revised: Protocol 1: BIWI dataset is used for training and evaluation in this protocol. BIWI dataset consists of 24 videos of 20 subjects with total 15 , 128 frames. Three random splits of 70%-30% images are used for training and evaluation. For the BIWI dataset, the yaw angle is in the range [ − 75 ◦ , 75 ◦ ] , the pitch is in the range [ − 65 ◦ , 85 ◦ ] , and the roll angle is in the range [ − 55 ◦ , 45 ◦ ] .

Original: Protocol 1: In this setting, the synthetic 300W-LP dataset is used for training, consisting of 122 ,samples.
Revised: Protocol 2: In this setting, the synthetic 300W-LP dataset is used for training, consisting of 122 ,samples.

Original: The trained network is tested on a real-world AFLW2000 dataset.
Revised: The trained network is tested on a real-world AFLW2000 dataset.

Original: Yaw, pitch, and roll angles are in the range [ − 99, 99] for both datasets. Angles in range [ − 180 ◦ , 180 ◦ ]with 360 quantization levels are considered for BEL. Protocol 2: BIWI dataset is used for training and evaluation in this protocol. BIWI dataset consists of 24 videos of 20 subjects with total 15 , 128 frames.
Revised: Yaw, pitch, and roll angles are in the range [ − 99 ◦ , 99 ◦ ] for both datasets.

Original: Three random splits of 70%-30% images are used for training and evaluation. For the BIWI dataset, the yaw angle is in the range [ − 75 , 75] , the pitch is in the range [ − 65 , 85] , and the roll angle is in the range [ − 55 , 45] .
Revised: Evaluation metrics: Mean Absolute Error (MAE) between the target and predicted values is used as the evaluation metric for this benchmark. MAE for a regression task is deﬁned as:

Original: 150 quantization levels are used for yaw and  pitch, and 100 quantization levels are used for roll.
Revised: [Equation] Here, N is the number of test samples, and P is the dimension of the regression task output. For head pose estimation, the dimension of regression output is three (i.e., yaw, pitch, and roll). y is the target, and ˆ y is the predicted label.

Original: Network Architecture and training parameters:
Revised: Network architecture and training parameters:

Original: We evaluate our approach on two models:
Revised: We evaluate our approach on two models:

Original: ResNet-50 and RAFA-Net.
Revised: ResNet-50 and RAFA-Net.

Original: With ResNet-50 (Direct regression and BEL), two runs with different random seeds for each combination of learning rate { 0 . 001 , 0 . 0001 , 0 . 00001 } and batch size { 8 , 16 } are used for hyperparameter tuning.
Revised: With ResNet-50, two runs with different random seeds for each combination of learning rate { 0 . 001 , 0 . 0001 , 0 . 00001 } and batch size { 8 , 16 } are used for hyperparameter tuning.

Original: For data augmentation, images are loosely cropped around the center in the training and testing datasets with random ﬂipping.
Revised: For data augmentation, images are loosely cropped around the center in the training and testing datasets with random ﬂipping.

Original: With RAFA-Net, we use the training parameters used in Behera et al. For data augmentation, we use the approach described inRAFA-Net Behera et al.
Revised: With RAFA-Net, we use the training parameters and data augmentation used in Behera et al.

Original: We refer to Protocol 1 evaluated with ResNet-50 as HPE1 , Protocol 1 evaluated with RAFA-Net as
Revised: We refer to Protocol 1 evaluated with ResNet-50 as HPE1 , Protocol 1 evaluated with RAFA-Net as

Original: HPE3 , Protocol 2 evaluated with ResNet-50 as HPE2 , and Protocol 2 evaluated with RAFA-Net as HPE4 .
Revised: HPE3 , Protocol 2 evaluated with ResNet-50 as HPE2 , and Protocol 2 evaluated with RAFA-Net as HPE4 .

Original: Table 21 provides a summary of the training parameters used with protocol 1.
Revised: Table 21 provides a summary of the training parameters used with protocol 1.

Original: Tableprovides a summary of the training parameters used with protocol2.
Revised: Tableprovides a summary of the training parameters used with protocol

Original: [Table]
Revised: [Table]

Revised: [Table]

Original: [Table]
Revised: [Table]

Original: Related Work Existing approaches for head pose estimation include stage-wise soft regression (Rothe et al., 2015; Yang et al.,  2019), a combination of classiﬁcation and regression (Mukherjee & Robertson, 2015;Ruiz et al., 2018)
Revised: Related work Existing approaches for head pose estimation include stage-wise soft regression (Yang et al., 2018; fsa, 2019), a combination of classiﬁcation and regression (Mukherjee & Robertson, 2015;

Original: [Table] and ordinal regression (Hsu et al., 2019).
Revised: Ruiz et al., 2018), and

Original: SSR-Net (Rothe et al., 2015) proposes the use of stage-wise soft regression to use the softmax values of classiﬁcation output to reﬁne the label.
Revised: 2018) proposes the use of stage-wise soft regression to use the softmax values of classiﬁcation output to reﬁne the label.

Original: FSA-Net (Yang et al., 2019) proposes extending stage-wise estimation to head pose estimation using feature aggregation.
Revised: FSA-Net (fsa, 2019) proposes extending stage-wise estimation to head pose estimation using feature aggregation.

Original: HopeNet (Ruiz et al., 2018) uses a combination of classiﬁcation and regression loss to train a model for head pose estimation whereas QuatNet (Hsu et al., 2019) proposes a combination of L2 loss and a custom ordinal regression loss.
Revised: HopeNet (Ruiz et al., 2018) uses a combination of classiﬁcation and regression loss to train a model for head pose estimation. Whereas, QuatNet (Hsu et al., 2019) proposes a combination of L2 loss and a custom ordinal regression loss.

Original: RAFA-Net (Behera et al., 2021) uses an attention based approach for feature extraction with direct regression.
Revised: RAFA-Net (Behera et al., 2021) uses an attention based approach for feature extraction with direct regression.

Original: We compare BEL with the performance of related work in Table 23 and Table 24.
Revised: We compare BEL with the performance of related work in Table 23 and Table 24.

Original: 95% conﬁdence intervals are given.
Revised: 95% conﬁdence intervals are given

Original: [Table]
Revised: [Table]

Revised: [Table]

Original: [Table]
Revised: [Table]

Original: D. F ACIAL L ANDMARK D ETECTION
Revised: D. F ACIAL L ANDMARK D ETECTION

Original: Facial landmark detection is a problem of detecting the ( x, y ) coordinates of keypoints in a given face image.
Revised: Facial landmark detection is a problem of detecting the ( x, y ) coordinates of keypoints in a given face image

Original:  Datasets We use the COFW (Burgos-Artizzu et al., 2013), 300W (Sagonas et al., 2013), WFLW (Wu et al., 2018)
Revised: [Table] Datasets We use the COFW (Burgos-Artizzu et al., 2013), 300W (Sagonas et al., 2013), WFLW (Wu et al., 2018)

Original: [Table]
Revised: [Table]

Original: and AFLW (Köstinger et al., 2011) datasets with data augmentation and evaluation protocols described in (Wang et al., 2020).
Revised: and AFLW (Köstinger et al., 2011) datasets with data augmentation and evaluation protocols described in (Wang et al., 2020).

Original: Normalized mean error (NME) and failure rate (f@10%) are used as evaluation metrics. .

Original: Inter-ocular distance normalization is used for the

Original: COFW, 300W, and WFLW datasets and bounding box-based normalization is used for the AFLW dataset.

Original: Data augmentation is performed by random ﬂipping, 0 .
Revised: Data augmentation is performed by random ﬂipping, 0 .

Original: 75 − 1 .
Revised: 75 − 1 .

Original: 25 scaling, and ± 30 degrees in-plane rotation for all the datasets.
Revised: 25 scaling, and ± 30 degrees in-plane rotation for all the datasets.

Original: We use 256 quantization levels for binary-encoded labels.
Revised: We use 256 quantization levels for binary-encoded labels.

Original: COFW
Revised: COFW:

Original: The COFW dataset (Burgos-Artizzu et al., 2013) consists of 1 , 345 training and 507 testing images.
Revised: The COFW dataset (Burgos-Artizzu et al., 2013) consists of 1 , 345 training and 507 testing images.

Original: Each image is annotated with 29 facial landmarks.
Revised: Each image is annotated with 29 facial landmarks.

Original: 300W This dataset is a combination of HELEN, LFPW, AFW, XM2VTS, and IBUG datasets.
Revised: 300W: This dataset is a combination of HELEN, LFPW, AFW, XM2VTS, and IBUG datasets.

Original: Each image is annotated with 68 facial landmarks.
Revised: Each image is annotated with 68 facial landmarks.

Original: The training dataset consists of 3 , 148 images.
Revised: The training dataset consists of 3 , 148 images.

Original: We evaluate the trained model on four test sets: full test set with 689 images, common subset withimages from HELEN and LFPW, challenging subset with 135 images from IBUG, and the ofﬁcial test set with 300 indoor and 300 outdoor images.
Revised: We evaluate the trained model on four test sets: full test set with 689 images, common subset with images from HELEN and LFPW, challenging subset with 135 images from IBUG, and the ofﬁcial test set with 300 indoor and 300 outdoor images.

Original: WFLW WFLW dataset consists of 7 , 500 training images where each image is annotated withfacial landmarks.
Revised: WFLW: WFLW dataset consists of 7 , 500 training images where each image is annotated withfacial landmarks.

Original: Full test dataset consists of 2 , 500 images.
Revised: Full test dataset consists of 2 , 500 images.

Original: We use test subsets: large pose (images), expression ( 314 images), illumination ( 698 images), make-up ( 206 images), occlusion (images), and blur ( 773 images).
Revised: We use test subsets: large pose (images), expression ( 314 images), illumination ( 698 images), make-up ( 206 images), occlusion (images), and blur ( 773 images).

Original: AFLW Each image has 19 annotated facial key points in this dataset.
Revised: AFLW: Each image has 19 annotated facial key points in this dataset.

Original: AFLW dataset consists of 20 , 000 training images where each image is annotated with 19 facial landmarks.
Revised: AFLW dataset consists of 20 , 000 training images where each image is annotated with 19 facial landmarks.

Original: The full test dataset consists of 4 , 836 images, and the frontal test set consists of 1 , 314 images.
Revised: The full test dataset consists of 4 , 836 images, and the frontal test set consists of 1 , 314 images.

Revised: Evaluation metrics: Mean Normalized Error (NME) between the target and predicted values is used as the evaluation metric for this benchmark.

Original: Network Architecture and training parameters:
Revised: NME for a regression task is deﬁned as: [Equation] Here, N is the number of test samples, and P is the dimension of the regression task output, i.e., the number of landmarks for facial landmark detection. y is the target, and ˆ y is the predicted label.

Revised: L is the normalization factor. .

Revised: Inter-ocular distance normalization is used for COFW, 300W, and WFLW datasets, and bounding box-based normalization is used for AFLW dataset.

Revised: We also report failure rate (f@10%) for some datasets.

Revised: The failure rate (f@10%) is deﬁned as the fraction of test samples with normalized errors higher than 0.1.

Revised: Network architecture and training parameters:

Original: We evaluate BEL by applying it on HRNetV2W18.
Revised: We evaluate BEL by applying it on HRNetV2W18.

Original: HRNetV2-W18 feature extractor’s output is 240 channels of size 64 × 64 .
Revised: HRNetV2-W18 feature extractor’s output is 240 channels of size 64 × 64 .

Original: For heatmap regression, a 1 × 1 convolution is used to get P heatmaps of size 64 × 64 , where P is the number of landmarks.
Revised: For heatmap regression, a 1 × 1 convolution is used to get P heatmaps of size 64 × 64 , where P is the number of landmarks.

Original: Since BEL-x predicts ( x, y ) coordinates directly we modify the architecture of
Revised: Since BEL-x predicts ( x, y ) coordinates directly we modify the architecture of

Original: HRNetV2-W18 to support direct prediction of landmarks.
Revised: HRNetV2-W18 to support direct prediction of landmarks.

Original: Figure 12 shows the modiﬁed architecture of HRNetV2-W18 for BEL-x.
Revised: Figure 12 shows the modiﬁed architecture of HRNetV2-W18 for BEL-x.

Original: The state-of-the-art approach with HRNetV2-W18 (Wang et al., 2020) uses heatmap regression, which minimizes the pixel-level loss between the predicted and target heatmaps.
Revised: The state-of-the-art approaches for facial landmark detection uses heatmap regression, which minimizes the pixel-level loss between the predicted and target heatmaps.

Original: We evaluate the applicability of BELon heatmap regression in Appendix A.
Revised: We evaluate the applicability of BEL on heatmap regression in Appendix A.

Original: In contrast, BEL-x predicts ( x, y ) coordinates directly with 256 quantization levels.
Revised: In contrast, BEL-x predicts ( x, y ) coordinates directly with 256 quantization levels.

Original: [Figure]
Revised: [Figure]

Original: We use two runs with different random seeds to decide the learning rate.
Revised: We use two runs with different random seeds to decide the learning rate.

Original: We consider learning rates { 0 . 0003 , 0 . 0005 , 0 . 0007 } and θ ∈  10 , 30  .
Revised: We consider learning rates { 0 . 0003 , 0 . 0005 , 0 . 0007 } and θ ∈ { 10 , 30 } .

Original: The value of θ is selected for NME, but

Original: Table 25 provides a summary of all the training parameters.
Revised: Table 25 provides a summary of all the training parameters.

Original: We refer to HRNetV2-W18 evaluated on
Revised: We refer to HRNetV2-W18 evaluated on

Original: COFW as FLD1 , on 300W as FLD2 , on WFLW as FLD3  and on AFLW as FLD4 .
Revised: COFW as FLD1 , on 300W as FLD2 , on WFLW as FLD3 , and on AFLW as FLD4 .

Original:  Related Work Facial landmark detection is an extensively studied problem used for facial analysis and modeling.
Revised: [Table] Related work Facial landmark detection is an extensively studied problem used for facial analysis and modeling.

Original: Common regression approaches for this tasks includes regression using MSE loss (Xiong & De la Torre, 2013; Lv et al., 2017), cascaded regression (Miao et al., 2018; Tzimiropoulos, 2015; Zhu et al., 2016; Sun et al., 2013b), and coarse-to-ﬁne regression (Sun et al., 2013a; Shizhan Zhu et al., 2015; Zhang et al., 2014).
Revised: Common regression approaches for this tasks includes regression using MSE loss (Xiong & De la Torre, 2013; Lv et al., 2017), cascaded regression (Miao et al., 2018; Tzimiropoulos, 2015; Zhu et al., 2016; Sun et al., 2013), and coarse-to-ﬁne regression (Sun et al., 2013; Shizhan Zhu et al., 2015; Zhang et al., 2014).

Original: State-of-the-art methods for this task learn heatmaps by regression to ﬁnd facial landmarks.
Revised: State-of-the-art methods for this task learn heatmaps by regression to ﬁnd facial landmarks.

Original: SAN (Dong et al., 2018) augments training data using temporal information and GAN-generated faces.
Revised: SAN (Dong et al., 2018) augments training data using temporal information and GAN-generated faces.

Original: DVLN (Wu & Yang, 2017), CFSS (Shizhan Zhu et al., 2015), LAB (Wu et al., 2018), DSRN (Miao et al., 2018) take advantage of correlations between facial landmarks.
Revised: DVLN (Wu & Yang, 2017), CFSS (Shizhan Zhu et al., 2015), LAB (Wu et al., 2018), DSRN (Miao et al., 2018) take advantage of correlations between facial landmarks.

Original: DAN (Kowalski et al., 2017) introduces a progressive reﬁnement approach using predicted landmark heatmaps.
Revised: DAN (Kowalski et al., 2017) introduces a progressive reﬁnement approach using predicted landmark heatmaps.

Original: LAB (Wu et al., 2018) also exploits extra boundary information to improve the accuracy.
Revised: LAB (Wu et al., 2018) also exploits extra boundary information to improve the accuracy.

Original: LUVLi (Kumar et al., 2020) proposes a landmark’s location, uncertainty, and
Revised: LUVLi (Kumar et al., 2020) proposes a landmark’s location, uncertainty

Original: [Table]
Revised: [Table]

Original:  visibility likelihood-based loss.
Revised: and visibility likelihood-based loss.

Original: Bulat & Tzimiropoulos (2016) proposes the use of binary heatmaps with pixel-wise binary cross-entropy loss.
Revised: Bulat & Tzimiropoulos (2016) proposes the use of binary heatmaps with pixel-wise binary cross-entropy loss.

Original: AWing (Wang et al., 2019) proposes adapted wing loss to improve the accuracy of heatmap regression.
Revised: AWing (Wang et al., 2019) proposes adapted wing loss to improve the accuracy of heatmap regression.

Original: AnchorFace (Xu et al., 2020) demonstrates that anchoring facial landmarks on templates improves regression performance for large poses.
Revised: AnchorFace (Xu et al., 2020) demonstrates that anchoring facial landmarks on templates improves regression performance for large poses.

Original: HRNet (Wang et al., 2020) proposes a CNN architecture to maintain high-resolution representations across the network
Revised: HRNet (Wang et al., 2020) proposes a CNN architecture to maintain high-resolution representations across the network

Original: [Table]
Revised: [Table]

Original: and uses heatmap regression.
Revised: and uses heatmap regression.

Original: The target heatmap is generated by assuming a Gaussian distribution around the landmark location.
Revised: The target heatmap is generated by assuming a Gaussian distribution around the landmark location

Original:  We compare BEL with related work in Table 26- 28.
Revised: [Table] We compare BEL with related work in Table 26- 29.

Original: 95% conﬁdence intervals are provided.
Revised: 95% conﬁdence intervals are provided

Original: [Table]
Revised: [Table]

Original: [Table] [Table] 
Revised: [Table] [Table] [Table]

Original: [Table]
Revised: [Table]

Original: D. A GE E STIMATION
Revised: D. A GE E STIMATION

Original: Age estimation aims to predict the age given an image of a human head.
Revised: Age estimation aims to predict the age given an image of a human head.

Original: Datasets We use the MORPH-II (Ricanek & Tesafaye, 2006) and AFAD (Niu et al., 2016) datasets for our evaluation.
Revised: Datasets We use the MORPH-II (Ricanek & Tesafaye, 2006) and AFAD (Niu et al., 2016) datasets for our evaluation.

Original: Cumulative Score (CS) and MAE are used as evaluation metrics.
Revised: Cumulative Score (CS) and MAE are used as evaluation metrics.

Original: We preprocess the MORPH-II dataset by aligning images ﬁrst along the average eye position (Raschka, 2018), then by re-aligning so that the tip of the nose is in the center of each image.
Revised: We preprocess the MORPH-II dataset by aligning images ﬁrst along the average eye position (Raschka, 2018), then by re-aligning so that the tip of the nose is in the center of each image.

Original: We do not preprocess the
Revised: We do not preprocess the

Original: AFAD dataset as faces are already centered.
Revised: AFAD dataset as faces are already centered.

Original: Afterwards, face images are resized to 256 × 256 ×and randomly cropped to 224 × 224 × 3 for training.
Revised: Afterwards, face images are resized to 256 × 256 ×and randomly cropped to 224 × 224 × 3 for training.

Original: For testing, a center crop of 224 × 224 × 3 is taken.
Revised: For testing, a center crop of 224 × 224 × 3 is taken.

Original: MORPH-II: This dataset consists of 55,608 face images with age labels between 16 and 70.
Revised: MORPH-II: This dataset consists of 55,608 face images with age labels between 16 and 70.

Original: The dataset is randomly divided into 39,617 training, 4,398 validation, and 11,001 testing images.
Revised: The dataset is randomly divided into 39,617 training, 4,398 validation, and 11,001 testing images.

Original: AFAD:
Revised: AFAD:

Original: This dataset consists of 164,432 Asian facial images and age labels between 15 and 40.
Revised: This dataset consists of 164,432 Asian facial images and age labels between 15 and 40.

Original: The dataset is randomly divided into 118,492 training, 13,166 validation
Revised: The dataset is randomly divided into 118,492 training, 13,166 validation, and 32,763 testing images.

Original: and 32,763 testing images.
Revised: Evaluation metrics: MAE (Equation 28) is used as the evaluation metric. We report Cumulative Score (CS θ ) for some datasets. CS θ is deﬁned as the fraction of test images with absolute error less than θ years.

Original: Network architecture and training parameters:
Revised: Network architecture and training parameters:

Original: We evaluate our approach on ResNet50.
Revised: We evaluate our approach on ResNet50.

Original: We perform two runs with different random seeds to determine the learning rate between [0 . 00001 , 0 . 0001 , 0 . 001] and use a batch size of 64 for all experiments.
Revised: We perform two runs with different random seeds to determine the learning rate between [0 . 00001 , 0 . 0001 , 0 . 001] and use a batch size of 64 for all experiments.

Original: We use ImageNet pretrained weights to initialize the network.
Revised: We use ImageNet pretrained weights to initialize the network.

Original: Full training parameters are described in Table 30.
Revised: Full training parameters are described in Table 30.

Original: We refer to our evaluation on MORPH-II as AE1 and AFAD as AE2 .
Revised: We refer to our evaluation on MORPH-II as AE1 and AFAD as AE2 .

Original: [Table]
Revised: [Table]

Original: Related Work Existing approaches for age estimation include ordinal regression (Niu et al., 2016;
Revised: Related work Existing approaches for age estimation include ordinal regression (Niu et al., 2016;

Original: Cao et al., 2020), soft regression (Rothe et al., 2015), and expected value ordinal regression (Pan et al., 2018; Gao et al., 2018).
Revised: Cao et al., 2020), soft regression (Yang et al., 2018), and expected value ordinal regression (Pan et al., 2018; Gao et al., 2018).

Original: OR-CNN (Niu et al., 2016) proposed the use of ordinal regression via binary classiﬁcation to predict the label.
Revised: OR-CNN (Niu et al., 2016) proposed the use of ordinal regression via binary classiﬁcation to predict the label.

Original: CORAL-CNN (Cao et al., 2020) reﬁned this approach by enforcing the ordinality of the model output.
Revised: CORAL-CNN (Cao et al., 2020) reﬁned this approach by enforcing the ordinality of the model output.

Original: SSR-Net (Rothe et al., 2015) proposed the use of stage-wise soft regression using the softmax of the classiﬁcation output to reﬁne the predicted label.
Revised: SSR-Net (Yang et al., 2018) proposed the use of stage-wise soft regression using the softmax of the classiﬁcation output to reﬁne the predicted label.

Original: MV-Loss (Pan et al., 2018) extended the soft regression approach by penalizing the output of the model based on the variance of the age distribution, while DLDL (Gao et al., 2018) proposed to use the KL-divergence between the softmax output and a generated label distribution to train a model.
Revised: MV-Loss (Pan et al., 2018) extended the soft regression approach by penalizing the output of the model based on the variance of the age distribution, while DLDL (Gao et al., 2018) proposed to use the KL-divergence between the softmax output and a generated label distribution to train a model.

Original: We compare BEL with related work in Table 31 and Table 32.
Revised: We compare BEL with related work in Table 31 and Table 32.

Original: 95% conﬁdence intervals are provided.
Revised: 95% conﬁdence intervals are provided.

Original: [Table]

Original: [Table]

Original: D. E ND - TO - END S ELF D RIVING
Revised: D. E ND - TO - END S ELF D RIVING

Original: We evaluate our approach on the NVIDIA PilotNet dataset and PilotNet model for end-to-end autonomous driving (Bojarski et al., 2016).
Revised: We evaluate our approach on the NVIDIA PilotNet dataset and PilotNet model for end-to-end autonomous driving (Bojarski et al., 2016).

Original: In this task, the steering wheel’s next angle is predicted from an image of the road.
Revised: In this task, the steering wheel’s next angle is predicted from an image of the road.

Original: We refer to these experiments as PN .
Revised: We refer to these experiments as PN .

Revised: MAE (Equation 28) is used as the evaluation metric.

Original: Dataset We use a driving dataset consisting of 45,500 images taken around Rancho Palos Verdes and San Pedro, California (Chen).
Revised: Dataset We use a driving dataset consisting of 45,500 images taken around Rancho Palos Verdes and San Pedro, California (Chen).

Original: We crop images to 256 × 70 × 3 then resize them to 200 × 66 × 3 .
Revised: We crop images to 256 × 70 × 3 then resize them to 200 × 66 × 3 .

Original: We randomly vary the brightness of the image between [0 . 2 × , 1 . 5 × ] , randomly ﬂip images [Table] and make random minor perturbations on the steering direction.
Revised: We randomly vary the brightness of the image between [0 . 2 × , 1 . 5 × ] , randomly ﬂip images, and make random minor perturbations on the steering direction.

Original: We use θ = 10 with 670 quantization levels for BEL.
Revised: We use θ = 10 with 670 quantization levels for BEL.

Original:  Training parameters We perform two runs with different random seeds to determine the learning rate between [0 . 00001 , 0 . 0001 , 0 . 001] and use a batch size of 64 for all experiments.
Revised: [Table] [Table] Training parameters We perform two runs with different random seeds to determine the learning rate between [0 . 00001 , 0 . 0001 , 0 . 001] and use a batch size of 64 for all experiments.

Original: Full training parameters are described in Table 33.
Revised: Full training parameters are described in Table 33.

Original: [Table]
Revised: [Table]

Original: Related Work End-to-end autonomous driving is a novel task that has become increasingly relevant due to the rise of self-driving vehicles.
Revised: Related work End-to-end autonomous driving is a novel task that has become increasingly relevant due to the rise of self-driving vehicles.

Original: The autonomous driving model’s task is to predict the future driving angle based on a forward-facing image from the perspective of the vehicle.
Revised: The autonomous driving model’s task is to predict the future driving angle based on a forward-facing image from the perspective of the vehicle.

Original: PilotNet (Bojarski et al., 2017) used a small, application-speciﬁc network to succeed within the time constraints of autonomous driving [Table]
Revised: PilotNet (Bojarski et al., 2017) used a small, application-speciﬁc network to provide good accuracy within the time constraints of autonomous driving.

Original: We compare BEL with the baseline PilotNet architecture in Table 34.
Revised: We compare BEL with the baseline PilotNet architecture in Table 34.

Original: 95% conﬁdence intervals are provided
Revised: 95% conﬁdence intervals are provided.

Original: [Table]
Revised: [Table]