Title: Improving Compositionality of Neural Networks by Decoding Representations to Inputs
Authors: Mike Wu
Noah Goodman
Stefano Ermon
Venue: NeurIPS 2021
Keywords: generative models
decoder
representations
out-of-distribution
compositionality

Abstract: In traditional software programs, it is easy to trace program logic from variables back to input, apply assertion statements to block erroneous behavior, and compose programs together. Although deep learning programs have demonstrated strong performance on novel applications, they sacrifice many of the functionalities of traditional software programs. With this as motivation, we take a modest first step towards improving deep learning programs by jointly training a generative model to constrain neural network activations to "decode" back to inputs. We call this design a Decodable Neural Network, or DecNN. Doing so enables a form of compositionality in neural networks, where one can recursively compose DecNN with itself to create an ensemble-like model with uncertainty. In our experiments, we demonstrate applications of this uncertainty to out-of-distribution detection, adversarial example detection, and calibration --- while matching standard neural networks in accuracy. We further explore this compositionality by combining DecNN with pretrained models, where we show promising results that neural networks can be regularized from using protected features.

TL;DR: Using generative models, we jointly optimize neural network activations to decode back to inputs, enabling a form of compositionality in neural networks that is useful for real-world applications.

Review discussion:
Model interpretability is an important topic, and the difficulty of debugging is indeed one weakness comparing neural networks to traditional programs.
Therefore, in general this work is well-motivated, and it demonstrates some pretty interesting results on interpreting neural network activations.
However, my main concern of this work is that simply applying this framework to an MLP model does not seem convincing. Due to the simplicity of the model architecture, the experiments are only conducted on relatively simple classification tasks, and the baseline performance is not great. What are the main difficulties of applying this framework to convolutional neural networks and/or recurrent neural networks? I wonder whether DecNN/ReDecNN still performs well when applied to more complicated neural network architectures.
Related to the fact that the implementation is based on a simplified model architecture, the authors did not empirically compare with recent OOD detection and calibration works, e.g., [20] [22] [23] [24].
Therefore, it is hard to tell whether ReDecNN indeed achieves a better trade-off between the classification accuracy and the OOD/calibration performance. The results in Table 5 show that even if the model simply utilizes a better model to improve its own performance, the OOD/calibration performance decreases notably, and in some cases the performance becomes worse than most or all baselines. This kind of indicates that the framework might not fit models that are able to better classification accuracies.
The results show that in general ReDecNN has lower classification accuracy than DecNN. I suppose the recursion depth D could be an important hyper-parameter for the performance. Do you have any ablation study that evaluates the performance of different D? Does increasing D generally improve the OOD detection/calibration results but decrease the classification accuracy?
While the discussion in Section 6 about fairness is interesting, I think the training objective is not directly related to the ReDecNN architecture. As we have seen, ReDecNN itself does not improve fairness. Have you tried applying the new term in Equation (8) to standard neural networks? I suppose you can simply achieve similar fairness results in this way.
No
Thank you for the comments and questions. We hope to address the main concerns below but please tell us if you would like elaboration.
> Concern that the framework is limited to an MLP.
In the general comment, we show preliminary results extending to a convolutional network.
> Authors did not compare with recent OOD detection and calibration techniques.
We wish to clarify that we do not claim our approach to be state-of-the-art for OOD or calibration.
Instead, the goal is to demonstrate the potential for a new neural network.
In particular, one where uncertainty measures provide OOD detection and calibration without any additional downstream computation. In this computation-free sense, I do not think recent OOD detection algorithms (e.g. ODIN [1]) nor calibration algorithms (Platt scaling, histogram binning, etc.) are the right baselines.
Instead, we chose to compare to other probabilistic classifiers where uncertainty can be used to do OOD and calibration w/o additional compute, such as Bayesian neural networks and Monte Carlo dropout.
[1] Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
> Does Table 5 indicate that the framework might not scale to larger models?
To clarify, Table 5 shows the effects of using a ResNet as a pre-trained classifier in the last term of Eq. 8. It shows that using more powerful models like a pretrained ResNet (which we do not expect to have good uncertainty and is known to be more poorly calibrated than a simple network) to teach your model results in less robust predictions. Given the ResNet is more expressive than the main classifier, this amounts to showing the opposite of regularization.
Critically, we emphasize this is different from using a more powerful classifier as the main neural network (i.e., scaling to larger models), which we show results for in the general comment above.
> Ablation study on the effect of depth? Does higher depth increase OOD detection?
In conducting the experiments, we did try a few different depths and found increased misclassification accuracy (on uncertainty scores) with higher depth, though with decreasing marginal gains (i.e., smaller gains for additional depth). For the final draft, we will add a more careful study on the effect of depth for different downstream tasks (OOD detection included).
> Is the fairness objective related to ReDecNN?
For clarification, the new term in Eq. 8 is specific to a ReDecNN since it has $m(g(h))$ i.e., the reconstructed image is fed to a pretrained classifier for the protected attribute. This is critical because this is how the gradients for this additional term affect the main network (ReDecNN) parameters. If we just did this with a standard neural network, the term would be $p(m(x) | 1/K 1_k)$ which is independent of the network parameters ($m$ has frozen weights).
Thank you for your additional results showing the computational costs and preliminary experiments using convolutional layers.
I agree that this work proposes an interesting new model architecture.
However, I am not convinced that this design is practically useful in its current state. The results show that this model does not scale up well to more challenging tasks, it is not very compatible with modern architectures, the computational overhead is high, and there is also no strong evidence that the model achieves better performance according to other metrics, like calibration and out-of-distribution detection.
Therefore, I still lean towards rejection at this point.
Thank you for the comment! We believe the results, especially the new results on CNN architectures, show that the model scales to neural networks beyond MLPs alone. Would you be able to clarify what you mean by lack of compatibility to modern architectures?
In theory, you could replace CNN blocks with residual blocks, for instance.
The OOD and calibration results consistently showed the method outperforming baselines, such as MC dropout and ensemble networks. We do not believe it is our burden to be state-of-the-art for calibration or OOD, but rather to provide a method where uncertainty is useful for this, which we believe the experiments show.
**Originality**: The idea of reconstructing the input from hidden activations seems old and used extensively in autoencoders. I didn't quite get the analogy to software 1. 5. What does it mean to obtain the decodable representation of the same "type" as input? I can see that the space of representations can be the same but there is nothing in the objective that forces the regenerated input to be the same "type" as the original input, especially if you take the software analogy where types have specific meanings ( like keywords, identifiers).
I felt that the analogy to software was unnecessary because the main idea doesn't bear any connection or even takes motivation from the software domain. I felt the invocation of software terms like *probing our “variables” (i. e. activations) for meaning, constraining model logic with “assertions”, and composing neural network modules together* (Line 60-61) was just a fancy way of motivating the work and somehow showing that they are doing something different from previous work.
**Technique**: I appreciate that the authors have mentioned this in limitations that their technique has a high computational cost. I would some analysis of the same though. I have some questions about some of the choices that the authors made in the formulation.
For example, in Equation (4), what is the need for reconstructing the input from *every* hidden layer and then taking a mean of that and adding to the objective? Essentially, we are directing the NN here to decode the input from every hidden layer in the NN, which places an additional burden on the NN. Imagine a big NN with multiple layers ( which will be used for slightly complex datasets like Imagenet), it will be extremely hard computationally to perform training. I understand that this gives the ability to probe every layer in the NN but at a huge computational cost thereby limiting the scalability of the approach to bigger models (especially if the recursive decodable NN in Equation 5 is used - which is the model that performs better in experiments).
**Experiments and Results**: I like the use-cases where the paper has shown results. To me, Section 5.2 and Section 6 were quite interesting. In Section 5.2, though I fail to understand why using a ResNet-18 pretrained classifier was worse than both without using any pretrained model or using a linear classifier?
Also, in Table 3, how were the attributes for OOD ( wearing a hat, long hair, bald ) chosen. Since in Table 2, MC is better than ReDecNN except in CelebA, choosing only CelebA for results in Table 3 seems unfair to other baselines. Similar question for attributes listed in Table 6. Were these results cherry-picked to show the superiority of ReDecNN over other techniques? As mentioned previously, I would also like to see an analysis of computational complexity of all the methods and the tradeoff with performance and interpretability. Anouther major concern I have related to the results is that the authors have chosen very simple datasets (MNIST, Fashion, CelebA). I feel like to evaluate the effectiveness of the proposed formulation, experiments on ImageNet or other datasets ( with higher image resolution and more complex) are needed.
**Clarity**: The paper is clear and well written. I felt that there are too many results though that may divert the attention from the main claims (like Section 7) that could have been placed in the Appendix.
No
We appreciate the comments and suggestions! We hope to address the questions below. Please let us know if we can elaborate on any area.
>  Idea of reconstructing inputs has been explored in autoencoders.
Although our approach is reminiscent of autoencoders, we note a few important differences:  While autoencoders only reconstruct the final hidden layer, DecNN has a reconstruction for every single hidden layer. This, and because DecNNs have a supervised objective (which autoencoders do not have) critically changes what information is captured in the hidden layers.
Whereas the autoencoder succeeds by recovering the input through a lossy channel, the DecNN succeeds by “interpolating” between the input and output. We show this in Section 4, where we find reconstructions to discard information unrelated to the supervised task (e.g. if we are predicting whether a person is bald or not, the face, gender, skin color of the reconstruction may vary throughout layers). This is not something the autoencoder does.
Finally, we point out that the applications of autoencoders are very different from the DecNN. The former does not measure uncertainty and is not used for calibration or composition.
> It would be beneficial to quantify the computational cost of the algorithm.
Please see the general comment above for reports of cost. We will include the table in the limitations section of the main text.
> What is the need for reconstructing the input from every hidden layer and then taking a mean of that and adding to the objective? Does this make it hard to train large models with many layers?
Good question! This is an important clarification to make as it is fundamental to the approach. The goal is to train neural networks with decodable activations.
As such, it would not be sufficient to only reconstruct the last layer as we are not trying to autoencode.
Rather, the goal of reconstructing from every layer to constrain the activations to be decodable i. e. , have the same “type” as the input (e. g. images). This way, we can do interesting things with the decoded activations, such as visualize them, compose a network with itself by feeding the decoded image as input to get uncertainty measures, or compose a network with other classifiers, or more easily constrain what a network has learned by working with decoded images (rather than arbitrary vectors).
Every method has its advantages and disadvantages. We hoped to convey in Section 9 that the tradeoff for optimizing for decodable activations is computational cost. It is more costly to optimize this model than a standard neural network but less costly than a naive ensemble. We were able to fit MLPs with 10 layers in a single Titan GPU. We also show initial experiments with convolutional networks above. As this method matures with future research, we are hopeful that the computational cost may decrease.
> Why is using a ResNet pretrained classifier worse than using any pretrained model?
Note that using the ResNet classifier has higher absolute accuracy than without using any pretrained model and when using a linear classifier, which you might expect. The ResNet pretraining did result in a drop in calibration and uncertainty experiments (OOD, adversarial, etc.): we believe this to be because a more powerful model overfits to the training dataset, resulting in a less robust classifier.
> How were the attributes for OOD ( wearing a hat, long hair, bald ) chosen?
Here was our process. In CelebA, some features are trivial in that there are mostly negative or positive labels. We ignored those as in prior work [1,2]. For the remaining 18 features, we arbitrarily chose three. These were not cherry picked, which we hoped to convey by showing three different ones.
[1] Invertible conditional GANs for image editing.
[2] Multimodal Generative Models for Scalable Weakly-Supervised Learning.
> Since in Table 2, MC is better than ReDecNN except in CelebA, choosing only CelebA for results in Table 3 seems unfair to other baselines.
To clarify, you could not do the experiments in Table 3 for MNIST or FashionMNIST: those datasets do not have explicit attributes. We included Table 3 as an extension to the results in Table 2 for CelebA. OOD based on attributes is a much harder problem than separating CelebA from MNIST or FashionMNIST since the OOD distribution is so close to the in-domain distribution.
> Were the results in Table 6 cherry-picked to show the superiority of ReDecNN over other techniques?
No, we chose the attractiveness and pale skin features prior to training any models. The choice was made to find two features that have confounding factors (e.g. correlation) and practical significance. The choice was arbitrary (as in you could have done the same with other feature pairs) but not made because of model performance.
> Datasets are too simple (MNIST, CIFAR, and CelebA)?
I disagree that every method must be tested on ImageNet.
Rather we are demonstrating a new neural network. In our experiments, I believe we sufficiently showed gains across many tasks and datasets that show promise in the approach.
Of course, scaling the method is important as well but we view it as a separate contribution for future work, and not necessary for the first paper introducing the method.
> Generalizing to other architectures may not be straightforward.
This is a fair point although we view our work as a proof-of-concept of what you can do with decodable activations in neural networks, not an exhaustive proof. In the general comment, we show preliminary results extending to a convolutional network.
Thanks a lot for your detailed response as well as for showing computational cost analysis and results on CNN.
However, my concerns over both these points remain.
Besides, the authors have not answered my other concerns related to invoking the software terminology and what does "type" mean here exactly.
(a) *Computational complexity*: What exactly is the ensemble in the results? The results do provide support to my original point that the approach is computationally quite expensive, and I have serious concerns over the scalability of the approach to architectures that are used for real-time applications.
(b) *Large architecture*: Just replaying the linear layer with a CNN doesn't have it complex, especially since CNN have weight sharing. I would have expected results on at least a network that is twice in big as the current one ( in terms of the number of layers since that is the factor on which your approach directly depends).
In fact, I will suggest varying the network complexity and then plotting the performance tradeoff with the number of layers and added computational complexity.
(c) *Resnet results*: I do not necessarily agree with your point here.
On the contrary, in recent papers, it has been shown that pre-trained models help in generalization.
I do not agree that if one is proposing a new technique, it is fine to show results in extremely simplistic settings. As pointed out by both me and Reviewer csH8, the results shown in the paper are just on MLP with 8 layers. Almost no systems of practical importance these days use this architecture. I am not convinced by the CNN results because of the reasons I mentioned above. My concerns of the technique being too computational complex to scale up to architectures of practical importance still remains. I believe that the performance gains over the baselines are not significant enough to justify the huge increase in computational complexity even for MLP, let alone other larger architectures.
I have decided to stand by my original score.
Thank you for the response and your time in reviewing! Please find our responses inline.
- What does "type" mean here?
Much of the proposed method (and applications) is dependent on the reconstructed output being analogous to the input (e.g. both are images). By "type", we mean the approach is set up such that a reconstructed output can be treated as input to the model, w/o any postprocessing.
- What is the ensemble?
The ensemble architecture uses exactly the same neural network as in other models, i.e., stacked MLP layers. We use an ensemble of the same size as the number of recursive depths we use in ReDecNN.
In practice, we use networks of 8 layers with 8 depths (or 8 in the ensemble.)
- No weight sharing in CNN experiments?
We were modeling our architecture after CNNs where weights are not shared, which is also commonly used in practice. Future work can explore replacing the CNN blocks with residual blocks, for instance.
- The experiments are on settings that are too simple.
We disagree that the experiments are too simple. We agree with the reviewer that there are more complex application settings e.g. large residual networks on ImageNet, but we do not consider CelebA to be too simple.
- Computational complexity is not worth the gains.
We agree that the method is computationally more expensive than a vanilla neural network -- we tried to be very transparent on this in the limitations section. From our point of view, we believe the cost is traded for a measure of uncertainty that is both very general and easy to use, but has advantages over existing methods like MCDropout or ensembling. We find this to be a meaningful contribution.
**Originality**
The idea of invertible generative models has been explored in earlier works (invertible flows).
However, this work is in a supervised setting where the activations are mapped to input space by training a generative model jointly with the classifier.
Also in this work, no prior is imposed on the activations.
Instead of relying on post-hoc computation, the proposed approach constrains the activations to be invertible.
The proposed recursive network ReDecNN presents a natural way for defining an ensemble network by randomly selecting activations (from the activation tree) to be fed to the network. This is especially appealing as the decoded activations retain only the task specific information.
**Quality**
* The paper is technically sound and the claims are supported by appropriate experiments
* The analogy with composition in traditional software seems a bit far fetched. The proposed composition by using the decoded activation as conduit is very different from the conventional composition operation.
For example, consider two hypothetical image to image networks f_flip and f_darken.   Now f_darken(f_flip(img)) will flip and darken the image.
However, with the proposed approach f_darken(g_phi(h)) will only darken the image since the decoded activation will invert the flip.
Admittedly, f_darken(f_flip(img)) is possible only when the input and output are of the same type. Comparing this conventional composition is confusing. Considering that “Compositionality” appears in the title, authors may want to clarify this aspect.
* The authors have gone to great lengths to demonstrate the usefulness of the recursive self-composition. All the claims are well supported by appropriate experiments. To demonstrate that the methods are applicable to other domains, authors have also presented results on the utterance and action recognition tasks.
**Clarity**
* The paper is well written and easy to read. The experiment results are presented well.
* Section 5. Line 143 talks about f1(f2(x)) but line 149 gives an example for f2(g_phi(h)). It will be better if the outer function is the same at both the places. It will also make sense to make it explicit that g_phi is a generative model trained along with f1 (and not f2).
* Apart from the computational cost, are there other reasons for parameter sharing across layers?
* Authors may want to emphasize the role of probing in error analysis.
**Significance**
* The idea of decoding the activations is novel. Since the decoded activations are in the input space, users can make sense out of them. The distinguishing feature of the approach is that no post-training computation is required to get the decoded activations.
* In addition to providing a regularizing effect, the decoded activations can also be used for data augmentation. This is especially useful since the decoded activations retain only the features that are important for prediction.
* The functionality of “constraining decodable computation” has a potential for significant impact. One can specify the protected information in input space and the method will take care that the protected information is not abused for solving the task.
No
Thank you for the comments and suggestions. We try to address the questions below but please let us know if we can elaborate further.
> Clarify what is meant by compositionality. Consider the $f_\text{darken}(f_\text{flip}(x))$ example.
This is an interesting question. Two different comments for the reviewer's thoughts:
1) It is not always the case that $g(h)$ will invert the flip. As shown in Section 4 (probing), the hidden layers “interpolate” between the input and the output. In the reviewer’s example, if the input is an image and the output is a flipped image, the early hidden layers will likely reconstruct the original image but the latter hidden layers will likely reconstruct the flipped image. If the user wishes to compose $f_\text{darken}(g(h))$ but preserve the flip it is up to them to choose which layer from $f_\text{flip}$ to use as the reconstruction.
2) More broadly, in this paper we focused on a specific form of compositionality where $f_1$ is a model and $f_2$ has the role of editing what $f_1$ is learning.
For instance, we explored (1) regularizing $f_1$ using a simpler model $f_2$ or itself via self-composition; (2) encouraging $f_1$ to ignore a protected feature with $f_2$ as a surrogate measure. That being said, we are excited about extensions to broader forms of compositionality (like $f_\text{darken}(f_\text{flip}(x))$) in future work. We could explore reusing what $f_1$ has learned to more efficiently train a second model $f_2$.
For instance, reusing information in the hidden states learned for a segmentation model to train a model more efficiently for detection (since the two tasks are very related).
> One should keep the notation consistent (line 143 vs line 149) and make clear that $g_\phi$ is a generative model, not another function.
Yes, great point. We will fix the notation to be $f_2(f_1(x))$ to keep it consistent and add text to specify that $g$ is a generative model inverting $f_1$. We are happy to add the following figure if reviewers agree: https://drive.google.com/file/d/1RKDNDxVDKBMuRjTp_Oza2nefCZ94Pr57/view.
> What is the reason for parameter sharing in g, aside from computational cost?
It was only for computational cost. As shown in the general comment above, this method has computational overhead and not sharing parameters would increase the burden a lot. We also wanted to be able to fit our models in a single GPU, which parameter sharing allowed.
Thank you for the response. You should make it explicit in the final version that the paper focuses on a specific form for compositionality. I don't think the proposed figure adds much value; The textual description is good enough.
Thank you for the response! We will make clear with text in the final draft that we focus on a particular form on compositionality.
The paper is well written, proposes a very new technique for training and analyzing the neural networks, and performs extensive evaluation with promising results. I thoroughly enjoyed the paper and definitely recommend it for acceptance.
The idea of learning a generative model to map the activations back to the inputs in-tandem is unique.
In fact, I was expecting the performance to degrade, but surprisingly the performance is as good as a standard network.
However, it is not clear to me how to evaluate the quality of the inversions (it can turn out that the generative modeling didn’t converge). Some of the images in figure 3 are promising and at the same time, some of them raise suspicions. For eg 3(b), in many of the intermediate images, it looks like the person has hair (instead of being bald). Can the authors comment on this?
Additionally, since ReDecNN performance is similar to DecNN, the expected “data augmentation” (line 162) is not really happening.
Nevertheless, the indirect evaluation through the uncertainties, OOD detection, and calibration is interesting.
Also, what is the rationale for using a deterministic generator model? What would happen with a non-deterministic generator model?
Overall, the technique presented in this paper is novel and promising and is definitely interesting to the community.
No
Thank you for the positive comments and questions. We hope to answer your questions below.
> How do you evaluate the quality of inverted images? For eg 3(b), why do the intermediate images include hair?
This is a good question! In Section 4, we randomly sampled images and qualitatively looked at their reconstructions through the network. We tried hard to optimize for enough epochs such that the loss of the generative model at least converged. More importantly, we access the quality of the inverted images through downstream tasks e.g. composition or constraining. If the reconstruction quality was really poor, such downstream computation would not be possible.
For the 3b example in particular, I would argue that these are not obvious examples of people with hair. There are certainly some images where a darker outline surrounds the head (fourth row) or that have background colors that resemble hair (bottom row), which the generative model often does when editing images to remove hair (i.e., shift it to the background) -- but largely, these generations look quite different than images of people who are not bald.
> ReDecNN and DecNN have similar performance, so is there benefit of data augmentation in training?
This is a good point and I think it is a big area for future research. For me, the fact that the model is producing these new images that are not in the training dataset but plausibly in the training distribution is interesting -- applications of what to do with these augmentations are not fully explored in this paper.
It is true that ReDecNN and DecNN have similar performance but that is already surprising since ReDecNN is solving a much more complex objective -- I could imagine ReDecNN doing much worse than DecNN in terms of overall accuracy, which is often the case when you make deterministic models probabilistic (e.g. MC-Dropout). One hypothesis is that the data augmentation is helping ReDecNN preserve similar accuracy to DecNN while producing useful uncertainty measurements.
> Why use a deterministic generative model vs a stochastic one?
We were originally excited about a stochastic generative model as well where different iid samples might generate semantically different reconstructions (e.g. man with two different kinds of hats).
However, when trying this, we found little diversity in generated content and a deterministic generator was both cheaper and easier to train. That being said, we still think this is exciting and could be another avenue for future research.

Meta-review:
role: PC
data: comment: UPDATE: The revision has been reviewed and the paper is officially accepted.

----

After extensive discussions among SACs, ACs, and the program chairs, we have decided to conditionally accept this paper. The primary concerns were around over-claiming, the interpretation of results, and the analogy to programming (which was viewed as a distraction from the main contributions of the paper).  There was some debate around whether these issues constitute a "fatal flaw" that should prevent the paper from being published at NeurIPS, but all agreed that these issues need to be addressed. We hope that the authors will take into account feedback from the reviewers and make the changes that were promised as part of the rebuttal.

Additionally, in order to be accepted, the following changes must be made in the revision:

Introduction:

L52: "This joint training optimizes the classifier not only to be highly performant, but constraints its activations to a manifold that the generative model is able to decode. Consequently, the networks entire computation is constrained to a 'decodable' vector space... " This is misleading and should be rephrased or removed; just because the activations at a given layer can decode the input doesn't mean that that information is being used by the downstream layers to actually make decisions.

L59: This paragraph should focus on what the paper actually demonstrates empirically; essentially remove the first sentence. 

Section 3: 

Highlight that equation (4) doesn't require that the reconstructed images $g_\phi(h_l)$ actually produce the same output y, so the reconstruction may not actually be highlighting the features that were important for f to make its decision in the first place.

Section 4: 

This section should be significantly rewritten to avoid making claims unsupported by the evidence. Specifically:

L116: "In practice, this theory leads us to expect 'useless' information..." This would be true if Equation (4) wasn't rewarding the network for keeping the useless information to help support the decoding task. 

L119: This paragraph is misleading and should be removed. While it is true that you can visualize the information at each layer, you cannot actually tell which of that information is actually being relied upon to make the decision. 

L130: This is the only paragraph in this section that states an empirical fact; the whole section should focus more on what this experiment objectively shows, but avoid making claims not supported by the observations made in this experiment.

L138: This paragraph again misrepresents what can really be inferred from the generated images. 

Section 6. 

The analogy with assertions is misleading as it implies a level of guarantees that is not supported by the evidence. The section should focus more closely on what the experiments actually demonstrate for this application. The attractiveness task should also be replaced with the "bald vs not bald", and "bearded vs not bearded" experiment presented in the rebuttal. 

Abstract: 

The abstract should be consistent with all the changes made to the rest of the paper. 

----

The reviewers found the idea of jointly training a generative model to additionally map neural network activations back to inputs quite interesting and novel. During discussions, a major concern about the approach that came up was regarding the computational cost of the approach and thereby its scalability for training more complex architectures and tasks. The additional results with CNN layers was helpful, but scalability still remains a challenge. However, there are enough interesting novel ideas and applications presented in the paper that could potentially open up many interesting future directions. It would be great if authors can incorporate the detailed feedback and clarifications from reviews and additional experiments in the final version of the paper.

Decision: Accept (Poster)