Identifying the Sources of Uncertainty in Object Classification
Authors: Luis Armando Pérez Rey
Berk İşler
Mike Holenderski
Dmitri Jarnikov
Venue:  (2021)
Keywords: Classification
Interpretability
Disentangled Representations
Uncertainty Estimation

Abstract: In image-based object classification, the visual appearance of objects determines which class they are assigned to. External variables that are independent of the object, such as the perspective or the lighting conditions, can modify the object's appearance resulting in ambiguous images that lead to misclassifications. Previous work has proposed methods for estimating the uncertainty of predictions and measure their confidence. However, such methods do not indicate which variables are the potential sources that cause uncertainty. In this paper, we propose a method for image-based object classification that uses disentangled representations to indicate which are the external variables that contribute the most to the uncertainty of the predictions. This information can be used to identify the external variables that should be modified to decrease the uncertainty and improve the classification.

Under review as a conference paper at ICLR 2021
IDENTIFYING THE SOURCES OF UNCERTAINTY IN OВJЕCT CLASSIFICATION
Anonymous authors Paper under double-blind review
ABSTRACT
In image-based object classification, the visual appearance of objects determines which class they are assigned to. External variables that are independent the object, such as the perspective the lighting conditions, can modify the object's appearance resulting in ambiguous images that lead to misclassifications. Previ- work has proposed methods for estimating the uncertainty of predictions and measure their confidence. However, such methods do not indicate which variables are the potential sources that cause uncertainty. In this paper, we propose method for image-based object classification that uses disentangled representations to in- dicate which are the external variables that contribute the most the uncertainty of the predictions. This information can be used to identify the external variables that should be modified to decrease the uncertainty and improve the classification.
INTRODUCTION
An object from the real world can be represented in terms of the data gathered from it through an observation/sensing process. These observations contain information about the properties of the object that allows their recognition, identification, and discrimination. In particular, one can obtain images from objects which represent its visual characteristies through photographs or rendering of images from 3D models.
Image-based object classification is the task of assigning a category to images obtained from an object based on their visual appearance. The visual appearance of objects in image is determined by the properties of the object itself (intrinsic variables) and the transformations that occur in the real world (extrinsic variables) (Kuikarni et 2015).
Probabilistic classifiers based on neural networks can provide a measure for the confidence of a model for a given prediction in terms of probability distribution over the possible categories image can be classified into. However, they do not indicate what variable contributes to the uncer- tainty. In some cases the extrinsic variables can affect the visual appearance objects in images in such way that the predictions are highly uncertain. A measure of the uncertainty in terms of these extrinsic features can improve interpretability of the output of classifier.
Disentangled representation learning is the task of crating low-dimensional representations of data that capture the underlying variability of the data and particular this variability can be explained in terms of the variables involved in data generation. These representations can provide interpretable data representations that can be used for different tasks such as domain adaptation (Higgins et al., 2017),com conttiooos learning (Achille et al., 2018), noise removal (Lopez al. 2018), and visual reasoning (van Steenkiste et al. 2019).
In this paper propose method for the identification of the sources of uncertainty in image- based object classification with respect to the extrinsic features that affect the visual appearance of objects in images by using disentangled data representations. Given image of an object, model identifies which extrinsic feature contributes the most to the classification output and provides information on how to modify such feature reduce the uncertainty in the predictions.

---

Under review as a conference paper at ICLR 2021
2
RELATED WORK
Achieving explainable results in predictive models is an important task, especially for critical ap- plications in which the decisions can affect human life such as in health, security, law and defence Barredo Arrieta et al. (2020). Even though deep neural networks provide successful results for im- age classification, their predictions can't be directly interpreted due to their complexity (Zhang & Zhu 2018). In order to solve this different approaches have been proposed to provide visual inter- pretability to the results such identification of the image regions that contribute classification Selvaraju et al. 2016).
The uncertainty of predictions provides an extra level of interpretability the predictions of a model by indicating the level of confidence in a prediction [Kndall & Gal (2017). There are different methods introduce uncertainty measures in classifiers which include bayesian neural networks, ensembles, etc.
Obtaining disentangled representations, that capture distinct sources of variation independently, is an important step towards interpretable machine learning systems Kim & Mnih (2018). Despite the lack of agreement on the definition, one description states that a disentangled representation should separate the distinct, informative factors of variations in the data Bengio et al. (2012).
Within deep generative models. disentanglement is achieved by using neural networks that approx- imate conditional distribution on the data and a set of unobserved latent variables. Particularly variational autoencoders (VAEs) Kingma Welling (2014) are heavily favored due to their abil- ity to model joint distribution while maintaining scalability and training stability HHiginn et al.) (2016). Therefore most of the methods are based on augmentations on original VAE framework Higgins al. (2016); Kim & Mnih (2018); Kulkarni et al. 20155; Mathieu et al. (2018).
In image-based object classification the variables that explain the visual characteristics of objects in data can be divided into those which represent the inherent properties of objects and those which represent its transformations. This explanation has been explored in Kulkarni et al. (2015) by de- scribing the object's 's properties the intrinsic variables and the properties that describe the object transformations as the extrinsic variables.
Other work refers to similar sets of variables and their disentanglement under different names but representing similar concepts. Haamauucii et al. (2019) disentangles the variables corresponding ambient variables with respect object identity information on images. (Gabbay & Hoshen 2020) proposes the learning of disentangled representations that express the intra-class variability terms of the class and content. (Detlefsen & Hauberg 2019) proposes the disentanglement of the appearance and the perspective. Separate the identity of cars from their pose (Yang et al.) 2015).
3 SETTING
Consider a dataset of images that have been generated from the observations of an object and which should be classified into certain category. We will assume that this category depends only on the properties of the object itself and not on its surroundings.
probabilistic classifier assign each of the images a category. Usually the output of a neural network can't be directly interpreted in terms of the characteristics of the object have affected the confidence of the prediction. Disentanglement serves as a method to produce interpretable oowddimeniional data representations that separate the variables that describe the properties of the objects and their surrounding.
We use a neural network
The main idea is that one can train probabilistic classifier on disentangled low dimensional repre- sentations and identify which variables contribute the uncertainty of the classification.
3.1 PROBABILISTIC CLASSIFIERS ON IMAGES
A probabilistic classifier is a model which outputs a conditional probability density Py over the set of K E N possible categories Y (1, .., ,K} conditioned on an input image x E X. The value Pr|z (v) can be interpreted the degree of confidence the model assigns for image X
2

---

Under review as a conference paper at ICLR 2021
to be classified into category Y. We will be only considering throughout this work probabilissic classifiers that use deep neural networks to obtain the predictions.
One can train probabilistic classifier using dataset of labeled images. Given labeled datapoint I E X with true label y² E Y. the neural network's weights are optimized to reduce the cross entropy between the network's output distribution Pyje and the true label ys. The cross entropy loss corresponds to,
L( v+)=- =- Σ
oy. log (u),
(I)
DEY
with oy.y the Kroenecker delta.
One can measure the degree of uncertainty the output by calculating the entropy of the output distribution Py for a given image. Higher entropy corresponds to a higher uncertainty. The entropy calculated as,
H(Pr - Pri (v)
DEY
Even though possible to provide a degree of uncertainty to the estimates of a probabilistic classifier trained on images, these estimates do not indicate which true generative variables that describe the image have contributed to the uncertainty. In this paper we assume that those sources of uncertainty are due the extrinsic variables that participate in the data generation process.
3.2 DATA GENERATION: INTRINSIC AND EXTRINSIC VARIABLES
We will assume that the underlying generative process for our data can be modelled by the joint probability distribution Px xv over the data space X and set of true generative variables V. The true variables condition the data generative process and represent the properties of the real world. In our case, we will consider that these variables can be divided into two sets V = v(I x V(E) that represent the intrinsic and extrinsic variables respectively.
The intrinsic variables are those which represent the properties of the object itself (e.g. its color, shape, size, material), while the extrinsic variables represent external factors (c.g. the light condi- tions or the relative position and orientation of the object with respect to the observer that generates the image). Intrinsic variables are invariant the transformations described by the extrinsic vari- ables.
The generative process consists of the independent sampling of an intrinsic variable v Pvtn and an extrinsic variable v( Pyom.. Those generative variables together determine conditional distribution over the data space from which data point ~ (xI((()),(E)) is sampled.
We assume that the intrinsic variables are independent of the extrinsic variables. During data gen- eration both variables are combined to produce the visual appearance of the objects following the formula for the joint probability density
(xxv (()))
(b()) Pv (v)()
We assume that the set of intrinsic and extrinsic variables can be separated into finite product of M,, Mg E N variables such that V V(()) x... v-()) Mr and y(E) v(E) x
V(E) The Mg
total number of true generative variables is then M = M, Mg.
3.3 DISENTANGLEMENT FOR INTERPRETABILITY
Learning disentangled representations is the task of creating interpretable low-dimensional data rep- resentations that separate the information about the variables that are involved in the generation of the data (Locatello et al. 2018). There is no common agreement on the definition of a disentan- gled representation. However, two properties have been proposed for disentangled representations that will be useful for our goal of producing interpretable measures uncertainty in classification predictions.
3

---

Under review as a conference paper at ICLR 2021
Modularity: No more than single dimension of the data representation should encode information about a true generative variable.
Compactess: Each of the true generative variables is encoded by a single dimension of the data representation.
If a data representation is both modular and compact, we can identify for each true generative vari- able its corresponding data representation's dimension that has all information about it. Consider Z Z1 x Zz
Zp as D-dimensional data representation space. One can measure the compactness of learned representation by measuring the mutual information between a data repre- sentation dimension Z, and true variable Vn, as
I(Pz:: ( Pz, & Pv..)
disentangled representation modular and compact if for each generative variable Vn there a unique latent dimension Zi such that Z(Pz,, Pv...) o 0 and I(Pz, Pv..) = 0 for all j * i. If a disentangled data representation is obtained which fulfills both modularity and compactness then can separate the latent dimensions in such a way that there is a decomposition of the latent space Z = z x Z into an intrinsic and an extrinsic set of latent variable dimensions. This would mean that probabilistic classifier trained on the latent variables (y)z(y) = Pr|za0 (y) would only depend on the intrinsic variables.
However, has been proved (Locatello et al. 2018 Mathieu et al. 2018) that without supervision disentanglement cannot be achieved. Therefore might not be able to achieve perfect modularity or compactness. This means that some information about the true intrinsic variables might be con- tained in the extrinsic latent dimensions such that there is dependency of the uncertainty in those variables as seen in Section
3.4 PROBABILISTIC DISENTANGLEMENT
Probabilistic methods for disentanglement propose generative distribution that approximates the true generative process in terms of set of unobserved latent variables that serve the data rep- resentation space Z. For simplicity we will assume that corresponds to a D-dimensional vector space, e.g. = Rd. The generative probability density function Pxxz over the data space X and latent space can be expressed as
Pxxz(I, 2) = (2))).
(4)
In order to provide a good approximation to the true generative model, the data distribution de- termined by the true model and the proposed in terms of the latent variables should match. This is expressed via the marginalization of the generative probability densities over the set of true generative variables and latent variables respectively, i.e.
Px(x) = / Px (v / (2))(a)
(5)
One problem that the integral above is in most cases intractable. There are different approaches solve this problem. In particular, variational inference offers method for approximating to the target true probability density (Blei et al. 2018) by means an approximation to the true posterior distribution.
The variational autoencoder (Kingma & Welling 2014) implements variational inference using an encoding neural network that calculates the parameters of the posterior approximate Q21² and the distribution Px also called the decoder distribution by using neural networks. The neural network weights are optimized by maximizing the lower bound of the latent variable model distribution over the data Px.
Given data point X and the approximate posterior Q2 is possible define a function over the data space z X + Z such that for data point x E X its representation in the latent space is given by
[xx)
B2- [=].
4

---

Under review as a conference paper at ICLR 2021
3.5 DISENTANGLEMENT FOR PROBABILISTIC CLASSIFIERS
A disentangled representation can be used to train probabilistic classifier and provide interpretabil- ity to the results in terms of each of its latent dimensions. Since the information about the true gen- erative variables is encoded in the separate latent variables one can estimate the uncertainty of such classifier and measure the change the uncertainty by small perturbations the latent dimension.
We can train probabilistic classifier with respect to the latent variables i.e. ( and estimate the uncertainty of a prediction as H (=)))))))) If this probabilistic classifier is trained with a neural network, we can obtain the gradient of the entropy with respect to the latent variables V H (a))))) via backpropagation. The gradient indicates the direction in the latent space that leads lower entropy in the prediction, i.c. to a latent variable with lower uncertainty.
For given image x E X, is possible to find the latent variable that would decrease the uncertainty by using the function z² : X + Z defined by
z" =x(x) =(x) - V
where corresponds to the size of the step taken in the direction of the negative gradient. Al- ternatively, it is possible to estimate the latent dimension that provides the highest change in the uncertainty of the prediction. We can evaluate the new latent variable obtained by taking step along only the d'thh latent dimension as
<x)x) = =(r) OH (P(II)))))
Eds
(8)
Ozd
with éd the d-th canonical basis vector. Then we can evaluate Equation 8 for each of the dimensions that corresponds the extrinsic variables to find which is the one that provides the highest decrease in entropy
2" (z) arg max H (PY)))()) (())))))))
2(EZ(E)
The generative variable associated with the latent dimension that produces the highest decrease in entropy can be considered as the one that contributes the most to the entropy.
4 EXPERIMENTS & RESULTS
In order to test the idcas presented in the previous section we used a synthetic dataset where we could have control
set of extrinsic and intrinsic variables. In this case we decided use dataset of images generated from 3D models where we could generate large amounts of data by varying different extrinsic variables.
Modelnet40 Aligned Dataset In order to investigate whether the disentangled representations of data can be useful to identify the sources of uncertainty we have used a synthetic dataset where we have control over most of the variables involved in the data generation. In particular we generate a dataset of rendered images obtained from 3D models within the Modelnet40 aligned dataset which contains 3D models from 40 different classes. Each 3D model is aligned to a certain predefined orientation which is unique to each class. Images were generated from only the car class by ren- dering of the cars for different configurations of the camera and object properties: relative azimuth and elevation of the camera with respect to the car, the light intensity and location within the scene, see Figure For the training set 40 car models were used and 8 were used for validation and for test. We have manually labeled each of the cars in each of the datasets into four categories for classification: SUV, sport, sedan and hatchback.
For each 3D car model We generated 6 images by changing four extrinsic variables (azimuth, elevation, light intensity, location of light) and one intrinsic variable (color) over 6 possible values.
Light Intensity: Indicates the intensity of the light source used in Watts per square meter W/me m/2 Values are from (0.5, 1.0, 1.5, 2.0,
5

---

Under review as a conference paper at ICLR 2021
Figure I: Diagram showing the changes in the extrinsic generative variables (light intensity, light location, elevation, azimuth) used to generate the dataset images rendered from Modelnet40.
2. Light Location: Indicates the position of the light source along axis, its values {-3, 1,,,,
3. Elevation: Determines the angular elevation of the camera with respect to the ground, the values used are {18.4, 22.6", 26.5", 30.25", 33.77, 36.9)).
4. Azimuth:
Indicates the relative angle of rotation of the camera with respect the front of the car when the camera is rotating with respect to an axis perpendicular to the floor, the values taken are {0", 36", 72"; 108", 144", 180%).
5. Color: This variable corresponds to an intrinsic feature of the car. Different hues were chosen for each car: green, cyan, blue, magenta, red and yellow.
Mutual Information
21
Q 2
22
Px|=
2D
x Pxxv
Py|
H ((r)))())
Clacsification
Performance
Figure 2:
Diagram representing the encoding of car images into latent representations by using DC-IGN encoder. The latent variables disentanglement is measured quantitatively via mutual infor- mation and qualitatively by doing latent traversals. The output classifier's 's uncertainty is calculated H H(P (()))()) based its gradient it is possible propose image that reduces the uncertainty by selecting the appropriate generative variables.
4.1 DISENTANGLED REPRESENTATIONS
We train Deep Convolutional Inverse Graphics Network (DC-IGN) (Kulkarii et al. 2015) since provides a method for obtaining disentangled representations by training a variational autoen- coder Kingma & Welling (204)) with batches of images where only only generative variable changes across the images. DC-IGN takes advantage of the batches to enforce the separation of the information of each generative variable into separate latent dimensions.

---

Under review as a conference paper at ICLR 2021
The encoder network has 3 convolutional blocks in which a convolutional layer with kernel size of 5 and x 2 max pooling with stride 2 are used. Differently from the original implementation, we set the size of the latent space as 128. The decoder network consists of 3 convolutional blocks with a 2 x 2 upsampling layer each followed by convolutional layer with kernel size 7. The training is performed for a total of 100 epochs.
The disentangled representations were characterized by measuring the discretized mutual informa- tion in Locatello al. (2018). The results are shown in T.ble/4.2 latent traversals are presented in Figure 3 Since the latent variables are encouraged to only contain information about the generative variable that changes in each batch we can directly associate a latent dimension to each generative variable as it can be seen in the results. The latent variable associated to the target generative variable does contain the largest value of mutual information.
One important note about our approach. we proposed the use of DC-IGN disentanglement learning algorithm which is tailored for the separation of extrinsic features. However, it is possible to use any other disentanglement approach as long as there is small subset of data where each extrinsic variable is available. such case it is possible to measure the mutual information between each generative variable and latent dimension for the small dataset.
One can assign the latent dimension with the highest mutual information to each of the true gen- erative variables such that the gradient with respect to latent dimension indicates change in the corresponding generative variable.
4.2 PROBABILISTIC CLASSIFIER: DISENTANGLED REPRESENTATIONS
We train probabilistic classifier with the latent variables obtained from the encoder DC-IGN by using the corresponding car type labels. We train simple neural network with 2 dense layers with 150 and 50 neurons each respectively and output dense layer with one neuron per category and a softmax activation function. The model is trained with early stopping using as hyperparameter a minimum loss decrease of 0.001 and patience of 3 epochs. The classifier achieved 89% accuracy on both the training and the test set. We estimated the uncertainty of the estimations using the softmax output of the intrinsic classifier. The average value of the metrics accross all car types is precision 0.91, recall 0.89, and FI-Score 0.89.
We reasoned the validity of the predictive uncertainty by evaluating with the Expected Calibration Error (BCE) metric as described in Guo et al. (2017). The metric measures the expected error between the estimation confidences and the true class probabilities and indicates the discrepancies between the accuracy and the uncertainty measures in our case this gives value of 0.015 (lower is better calibration). Comparatively, as presented in Guo et al. (2017), ResNet 50 trained on Stanford Cars Krause et al. (2013) data set has 0.0430 ECE without any additional calibration method. Even with additional calibration enchancements the lowest ECE of Resnet 50 set is 0.0174.
Given that the intrinsic classifier is calibrated, the uncertainty of the classifier is estimated with the entropy of the softmax output. In Section [3.4] we mentioned that we expect the true extrinsic generative variables affect the uncertainty of clasifier. To justify this claim, we measured the entropy of the images predicted for the classifier and compared their distribution across the values of each extrinsic generative variable.
If the extrinsic generative variables do not affect the uncertainty predictions then across different values of the same generative factor the distribution of entropies should be the same across different values of the extrinsic variable. We used the Mann- Whitney U (MWU) test to test whether the Mann-Whitney
distribution of entropy between two different values of a generative variable are different. The null hypothesis for this test states that there is not statistically significant difference between two distributions. In Appendix A we show the log entropy distributions for each extrinsic variable with violin plots together with the p-values from the pair-wise comparison of MWU test.
Only for the tests between angle 36 and 72 of the azimuth, 22.6" and 26.5% of elevation and 2, for the light location the null hypothesis cannot be rejected with p-value larger than 0.05. In other words except for these particular combinations of extrinsic variables we conclude that the distribution of entropy values are statistically different from each other for the generative variables. This means that the uncertainty of the predictions is indeed dependent on the true extrinsic variables for the trained classifier.
7

---

Under review as a conference paper at ICLR 2021
Light 190b0
Ligh locaaton
Eleaaion
Figure 3: Examples of latent traversals across the latent dimensions correspond to each true genera- tive variable.
Table 1:
Mutual information values obtained between the corresponding latent dimensions of Z
associated to the corresponding true generative variables of V.
Light Intensity
Light Location
Elevation
Azimuth
Color
Light Intensity
1.230
0.104
0.010
0.019
0.008
Light Location
0,077
0.764
0,011
0.072
0.013
Elevation
0.071
0.022
1.562
0.024
0.007
Azimuth
0.098
0.067
0.011
0.938
0.010
Color
0.039
0.020
0.013
0.024
1.631
CONCLUSION
In this work we proposed the use of disentangled data representations to provide interpretability to the results of classifiers based on the set of extrinsic variables that affect the estimation of the intrinsic properties of objects.
We showed that for probabilistic classifier trained on the disentangled latent variables, the extrinsic variables affect the uncertainty of the predictions. Moreover we provide a method to modify the latent variables in order to identify the extrinsic variable that produces the strongest uncertainty in the classifier's 's predictions.
Future work will consider the exploration of methods react upon the sources of uncertainty in amage-based object classification by proposing actions to an agent that gethers data in order to modify the extrinsic variables that produce the uncertainty.
REFERENCES
Alessandro Achille, Tom Eccles, Loic Matthey, Christopher Burgess, Nick Watters, Alexander Lerchner, and Irina Higgins. Life-long disentangled representation learning with cross-domain la- tent homologies. Advances in Neural Information Processing Systems, (eM)-DecemMMdd)99733-- 9883, 2018. ISSN 10495258.
Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina. Richard Ben- jamins, Raja Chatila, and Francisco Herrera. Explainable Artificial Intelligence (XAI): Con- cepts, taxonomies, opportunities and challenges toward responsible AL Information Fusion, 58 (December 2019):82-115, 6 2020. ISSN 15662535. doi: 10. 20.10016..infuus.2019..0000 URL https:://iikiigghae
sevi www.veer.cm/retrrieeeee.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation Leaming:: A Review and New Perspectives. (1993);1-30, 6 2012. URL htt.:ttp.//axiwworr//aaaae// 5538
8

---

Under review as a conference paper at ICLR 2021
David M Blei, Alp Kucukelbir, and Jon D Mcauliffe. Variational Inference A Review for Statisti- cians. pp. 1 1-41,2200 2018.
Nicki Skafte Detlefsen and Søren Hauberg. Explicit Disentanglement of Appearance and Perspec- tive in Generative Models. (NeurIPS), 2019.
11881
Aviv Gabbay and Yedid Hoshen. Demystifying Inter-Class Disentanglement. In International Con- ference on Learning Representations, pp. 1-2, 6 2020. URL http://arxiv, www.eorggaaaoo /bg/abs/
1996..1770
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. 34th International Conference Machine Learning, ICML 2017, 3:2130-2143, 2017.
Ryuhei Hamaguchi, Ken Sakurada, and Ryosuke Nakamura. Rare event detection using disen- tangled representation learning. Proceedings of the IEEE Computer Society Conference Computer Vision and Pattern Recognition, 2019-June:9319 9327, 2019. ISSN 10636919, doi: 10.1109/CVPR.2219.0000
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. B- Vae: Learning Basic Visual Concepts With Constrained Variational Framework. Iclr 2017, (July):1-13, 2016. ISSN 1078-0874. doi:
10.1177/1080874403220553
Irina Higgins, Arka Pal, Andrei A. Rusu, Loic Matthey, Christopher P Burggess Alexander Pritzel, Matthew Botvinick, Charles Blundell, and Alexander Lerchner. DARLA: Improving Zero- Shot Transfer in Reinforcement Learning. 2017. URL http:://rxiiy http://...r//b//100000//
08475
Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? (Nips), 2017. ISSN 16653521. doi: 10.100/TDEE.20095221182.. URL
http://aartt.crrrr//////0....00
org/ 2ttpgg//abb11770009977
Hyunjik Kim and Andriy Mnih. Disentangling by Factorising. 2018. ISSN 21622701. doi: 10.
1364/CLEO.200 2009.. CMS6. URL URL
05983
Diederik P Kingma and Max Welling. Auto-Encoding Variational Bayes. In International Confer-
ence on Learning Representations (ICLR), 2014. ISBN 1312.6114v10. doi: 10.1051/0004-630
201527329.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3D Object Representations for Fine- Grained Categorization. In 2013 IEEE International Conference on Computer Vision Workshops, pp. 554-561. IEEE, 12 2013. ISBN 978-1-47999-322-77, doi: 10. 1109/CCCV.20013.77.. URL ieee. dww///occmmtt//75555500
http::/Leeexppeeee
Tejas D. Kulkarni, Will Whitney, Pushmeet Kohli, and Joshua B. Tenenbaum. Deep Convolutional Inverse Graphics Network. pp. 1-10, 2015, ISSN 10897550. doi: 10.1063/1.4914407,, URL
http://arxiv org. abs / 1t../553.00 03167
Francesco Locatello, Stefan Bauer, Mario Lucie, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem, Challenging Common Assumptions in the Unsupervised Learning of Disentangled Rep- resentations. pp. 1-33, 2018. URL./tttp:/aaxxxv wag.
12359
Romain Lopez, Jeffrey Regier, Michael Jordan, and Nir Yosef. Information constraints on auto- encoding variational Bayes. Advances in Neural Information Processing Systems, 2018-Decar (1):6114-6125, 2018. ISSN 10495258.
Emile Mathieu, Tom Rainforth, N. Siddharth, and Yee Whye Teh. Disentangling Disentanglement in Variational Autoencoders. 2018. URL http::ttt//aaat org/ abs/ 1812. 02833
Ramprasaath Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM Crad Visual Explanations from Deep Networks via Gradient-based Localization. pp. 1-23, 10 2016. doi: 10. 120007//1111110-1-111222281111 URL Bttp:t////axxxy doi
org/abs//611 /1000 02391 http:///wo...o...ooooo/o///////0

---

Under review as a conference paper at ICLR 2021
Sjoerd van Steenkiste, Francesco Locatello, Jürgen Schmidhuber, and Olivier Bachem. Are Dis- entangled Representations Helpful for Abstract Visual Reasoning? (NeurIPS), 2019. URL
http:://axiv..ort abs 1905 httbs//995.11500 12506
Jimei Yang, Scott Reed, Ming Hsuan Yang, and Honglak Lee. Weakly-supervised disentangling
with recurrent transformations for 3D view synthesis. Advances in Neural Information Processing Systems, 2015-Janua: -1w-Ja:an099-10000000 1099-1107, 2015. ISSN 10495258.
Quan-shi shi Zhang and Song-chun Zhu. Visual interpretability for deep learning: survey. Frontiers of Information Technology & Electronic Engineering, 19(1):27-39, 2018. ISSN 2095-9184. doi: 1631/FFTEE. 1700808. URL
700808
A UNCERTAINTY DEPENDENCE ON EXTRINSIC VARIABLES
In this appendix, the plots representing the distribution of uncertainty values of the probabilisticc classifier trained the disentangled latent variables together with the Mann-Whitney U test p- values are presented. The violin plots show the distribution of the uncertainty with respect to the true extrinsic variables and the p-value tables indicate whether pair of values for extrinsic variable produce statistically significant different uncertainty. If > 0.05 the uncertainty for two values of generative variable are similar.
Error Log Entropy Type Estimation per Elevation
Distribution of Estimation Log-Entropy Elevation
-0.5
-1.0
enoopy
-1.5
Log ogg-tntropy
-2.0
-10
-3.5
-12
Elevation
Elevation
Figure 4: Error-bat and violin plots for the elevation extrinsic feature
Error of Log Entropy Type Estimation
Distribution of Estimation Log-Entropy per Location
Log Entropy
Log Entropy
-12
2
1
Location
Location
Figure 5: Error-bar and violin plots for location extrinsic feature
10

---

Under review as a conference paper at ICLR 2021
Error Bars Log Entropy Type Estimation per intensity
Log Ent-Entrpp per Intensity
use
Log Entropy
Log Eng-Entropy
-10
-4
-12
0.5
100
1.5
2.0
2.5
300
Intensity
30
Figure 6: Error-bar and violin plots for the intensity extrinsic feature
Azimuth
0
36"
72
108
144
180
0"
0.000
0.000
0.000
0.000
0.000
36
0.000
0.190
0.000
0.000
0.001
72"
0.000
0.190
0.000
0.000
0,003
108
0.000
0.000
0.000
0.000
0,000
144"
0.000
0.000
0.000
0.000
0.000
180
0.000
0.001
0.003
0.000
0.000
Table 2: p-values of MWU test of azimuth latent variable
Table 3: p-values of MWU test for the location of elevation generative variable across different
values
Elevation
18.4
22.6
26.5
30.25
33.7
36.9
18.4
0.000
0.000
0.000
0.000
0.045
22.6
0.000
0.198
0.000
0.000
0.000
26.5
0.000
0.198
0.000
0.000
0.000
30.25
0.000
0.000
0.000
0.000
0.000
33.7
0.000
0.000
0.000
0.000
0.000
36.9
0.045
0.000
0.000
0.000
0.000
Table 4: p-values of MWU test for the location of lightning true generative variable across different
values
Location
-3
0.000
0.000
0.000
0.000
0.000
-2
0.000
0.000
0.000
0.142
0.000
-1
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
2
0.000
0.142
0.000
0.000
0.000
3
0.000
0,000
0.000
0,000
0.000
Table 5: p-values of MWU test for the intensity of lighting true generative variable across different values
Intensity
0.5
1.0
1.5
2.0
2.5
3.0
0.5
0.000
0.000
0.000
0.000
0.000
1.0
0.000
0.000
0.000
0.000
0,000
1.5
0,000
0.000
0.000
0.000
0,000
2.0
0.000
0.000
0.000
0.000
0.000
2.5
0.000
0.000
0.000
0.000
0.000
3.0
0.000
0.000
0.000
0.000
0.000


Reviews:
[{"review_id": "suMhjT8AxAl", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nMajor comments:\n-\tThe methods suggested does not apply to real-world classifiers:\no\tIt required training with strong labels only available in simulation\no\tClassification is done based on a low-dimensional intermediate representation of the ‘entangled space’, and its accuracy is probably inferior to what can be obtained with general CNNs (no comparison was made)\no\tI do not see how this can be applied to real-world competitive classifiers\n-\tThe findings are rather limited\no\t The method suggested for finding which extrinsic variable is able to reduce the prediction entropy is a) not well founded, and b) was not tested experimentally  \n-\tThere are presentation clarity problems, with non-sentences, broken references, etc..\n\nMore detailed comments:\n\n-\t“Separate the identity of cars from their pose (Yang et al., 2015).” – this is not a sentence.\n-\tThe probability estimates provided by neural networks are uncalibrated (not reflecting the real error probability)\n               o\tThis is not mentioned in the introduction, but is discussed in page 7\n-\tPage 4 “maximizing the lower bound of the latent variable model distribution” – what is a lower bound over a distribution? Lower bounds are defined for scalars, not distributions. This sentence is not clear.\n-\tPage 5: before equation7 the text reads “it is possible to find the latent variable that would decrease the uncertainty…”. However, the equation does not define a latent variable, not does the text in the 1-2 sentences following it. The equation defines a vector in R^M. It is not stated how it defines a latent variable\n-\tPage 5: The process described in equations 8,9 is essentially making a gradient step (of the entropy) in each of the M_E possible directions and then measures which step reduced the Entropy. This is a numerical re-estimation of the gradient. Why is it done? At least for small \\alpha, the direction most minimizing the entropy can be determined from the gradient directly (the coordinate in which the (negative of ) gradient is highest)\n-\tPage 7: The results are shown in Table 4.2, = 4.2 is a broken reference\no\tAlso: it is not clear how mutual information is calculated. It requires discrete variables, and the quantization details may be important. \n-\tThe classifier trained has a bottleneck at the  ‘entangled representation’ layer, and is hence probably is far from being optimal in terms of its obtained error\n-\tThe method suggested for estimating which extrinsic variables most affect the uncertainty was not applied experimentally (no results were reported)\n\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "qDBDFmOEIMc", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThe paper presents an approach that for every object identifies the factors that have a high impact on the models' uncertainty. The approach consists of i) disentangled representations ii) a classifier on the top of the trained representations iii) technique that select dimensions of representation of an objects' (factors), that impact the uncertainty of a model the most. The i) and ii) are done in a known way; The disentanglement is done via Deep Convolutional Inverse Graphics Network (2015), and a classifier is trained with a standard maximum likelihood approach.\nThe iii) suggests selecting a dimension that influences uncertainty the most, based on the gradient of uncertainty criteria (entropy of p(y|z), where z is embedding). In other words, the suggestion is to select the component of representation that \"provides the highest decrease in entropy\".\nThe experiments are done in a controllable setting, where the generative process is known and can be controlled. The model demonstrated the ability to learn disentangled representations, in the case when control of generative prosses is available. Specifically, it is required to generate/have objects were \"only one generative variable changes across the images\". However, the connection to uncertainty is weak.\n\nReview:\n\nI have very mixed feelings about the work. On one hand, the problem is interesting (perhaps novel) and deserves the attention of the community. On the other hand, the paper is half-baked, there are conceptual flaws, the evaluation protocol is weak, there is an incorrect interpretation of experiment results;  \n\nMy comment on novelty is the following: I would say that algorithmic novelty, is the weak point, however, bringing the attention of the community to important problems is not less important than new algorithms (and even is more important IMO). Taking into account that \"novelty\" is extremely subjective, I would say that there is nothing wrong with the novelty side. \n\nWriting: the paper is clearly written. \n \nConcerns:\n\n1) The identified factors do not proofed to be meaningful and useful. Pure identifying factors of uncertainty or reduction uncertainty of a model do not give us the full picture. The factors may be flawed in many ways: factors may be too dependent on random errors of the model but not to be semantically meaningful. Reduction uncertainty of a model is not always useful, as most models nowadays are often wrongly overconfident, more confidence is not necessarily good! It has not been proven that identified factors are connected to uncertainty. \n\n2) There is a noticeable connection between the proposed method and adversarial attacks. We can interpret a method as a way to \"trick\" a model to be more ceratin on adversarial modifications of embeddings. In this case, the results would not be useful.\n\n3) There are two flaws of experiments in section 4.2: i) the authors compare Expected Calibration Error between different datasets, these figures are not comparable, it is the same as compare accuracy between MNIST and ImageNet  ii) ECE is a biased estimate of true calibration with a different bias for each model, so it is not a valid metric to compare even models trained on the same data (see Vaicenavicius2019). Yes, ECE is the standard in the field, but it is the wrong standard that prevents us from meaningful scientific progress, so we should stop using it.\n\nOverall: The direction is interesting, however, the evaluation protocol needs to be more thoughtful. At the moment, it is impossible to verify the real performance of the method.\n\nSuggestions:\n\n1) Evaluate the identifying algorithm on downstream problems. For example, can we use these factors to collect additional data that will improve predictive performance better than uniformly sampled data (aka active learning)? There should be many more interesting settings.\n\n2) Evaluate the model on controllable uncertainty estimation setting: identify setting where we understand on which examples we expect the model to be uncertain (due to not enough data in a dataset in a certain region, etc.), validate selected factors.\n\n3) To use the squared kernel calibration error (SKCE) proposed in [Widmann2019] along with de facto standard, but biased ECE. The SKCE is an unbiased estimate of calibration. There might be some pitfalls of this metric that I'm not aware of, but the paper looks solid and convincing. Also, please put attention to Figure 83 in the arХiv version. \n\nEditing:\n- Citations: It is better to use \"authors (year)\" style when a citation is a part of a sentence---\"(Gabbay & Hoshen, 2020) proposes\" to \"Gabbay & Hoshen, (2020) propose\", and otherwise \"(authors, year)\" when a citation is not a part of a sentence \"on original VAE framework Higgins et al. (2016);\" to on original VAE framework (Higgins et al., 2016; .....).\n- \"only only\" typo in 4.1\n\n[Vaicenavicius2019] Juozas Vaicenavicius, David Widmann, Carl Andersson, Fredrik Lindsten, Jacob Roll, and Thomas B Schon. Evaluating model calibration in classification. AISTATS, 2019.\n\n[Widmann2019] Widmann D, Lindsten F, Zachariah D. Calibration tests in multi-class classification: A unifying framework. In Advances in Neural Information Processing Systems 2019 (pp. 12257-12267). https://arxiv.org/pdf/1910.11385.pdf", "extra_scores": {}, "extra_text": {}}, {"review_id": "bSGSOVecCX2", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to identify sources of uncertainty by disentangling representations in latent spaces for object classification tasks. Experiments on a synthetic dataset demonstrate that the proposed method can disentangle different extrinsic variables' contribution to the prediction uncertainty.  In addition, the authors propose to modify the latent variables to decrease uncertainty in the predictions.\n\n### Clarity\n#### Pros\n- This paper provides a clear discussion of the main contributions, assumptions, experimental settings. \n\n#### Cons\n- The experiment section needs more analysis. For example, how to explain the results in Table 1, which variable contributes most to the uncertainty and which contributes least?\n- The paper contains typos and grammatical errors and need to be proofread carefully.\n- Several questions needed to be addressed.\n\nQuestions:\n1. How to choose the latent representations? In real applications, there are usually more factors.\n2. It is not clear how the model in Figure 2 is trained. e.g. what is the loss function, do you need supervision on the latent variable.\n\n### Originality\n#### Pros\n- The paper proposes a  method to identify sources of uncertainty by disentangling representations in latent spaces in object classification tasks.\n\n#### Cons  \nThe contribution is unclear as the disentangling method is based on the DC-IGN method.  Moreover, it is unclear how to relate the imaging factors to the latent variables as multiple imaging factors can impact the same latent variable. \n\n### Significance\n#### Pros\n- It addresses an important problem; the problem of decomposition of the sources of uncertainty in model prediction is important and has not been sufficiently addressed in existing works.\n\n#### Cons\n- Entropy of the softmax is not a reliable criterion to capture uncertainty and, more importantly, other factors besides the imaging factors such as the input data density can also contribute to the uncertainty of the prediction. \n- The experiment is performed on synthetic data, which leads to limited evaluation conditions. As there is a distribution difference between synthetic data and real data, it would be more significant if the proposed method can be applied to real images.\n- The latent variables in the study are limited, only including light intensity, pose, color, etc. In real applications, there are usually many more factors such as background, occlusion, noise, resolution, etc. It would be better if the authors can study more factors or explain why specific factors are chosen.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The manuscript presents an approach for identifying sources of uncertainty in object classification tasks by disentangling representations in latent spaces.

Three reviewers agreed that the manuscript is not ready for publication. 
Some of the concerns are conceptual flaws, weak evaluation protocol, and an incorrect interpretation of experiment results.

There is no author response. 

Decision: Reject