Paper:
While neural networks can be trained to map from one specific dataset to another, they usually do not learn a generalized transformation that can extrapolate accurately outside the space of training. For instance, a generative adversarial network (GAN) exclusively trained to transform images of black-haired men to blondhaired men might not have the same effect on images of black-haired women. This is because neural networks are good at generation within the manifold of the data that they are trained on. However, generating new samples outside of the manifold or extrapolating "out-of-sample" is a much harder problem that has been less well studied. To address this, we introduce a technique called neuron editing that learns how neurons encode an edit for a particular transformation in a latent space. We use an autoencoder to decompose the variation within the dataset into activations of different neurons and generate transformed data by defining an editing transformation on those neurons. By performing the transformation in a latent trained space, we encode fairly complex and non-linear transformations to the data with much simpler distribution shifts to the neuron's activations. We motivate our technique on an image domain and then move to our two main biological applications: removal of batch artifacts representing unwanted noise and modeling the effect of drug treatments to predict synergy between drugs.

Review:
The authors present a way to transform data from a source distribution to have characteristics of a target distribution.
This is accomplished by applying a "NeuronEdit" function to the encoding of the input; this edited input is then decoded.
The NeuronEdit function is parametrized by the target distribution's statistics. The edit function does a sort of simple histogram matching, so that the ith percentile values of the source distribution's bottleneck representations instead become the ith percentile values of the target distribution's bottleneck representations.
Experiments are on CIFAR-10 and biology datasets (the latter of which are not my strong suit).
This paper is well-written and original. It is original because there are only a few works which directly manipulate the latent space (one example is latent space interpolation used to visualize GANs), and this is distinct from those.
The problem they aim to solve also has not received much attention, which enhances the novelty of this paper.
The presented method is simple and easy to implement, since the editing function is not learned but is instead deterministic. It is encapsulated in Equation 1.
The fact that the editing function is fixed may greatly hinder its flexibility and applicability.
In Section 3.1 and Figure 1, we are shown that NeuronEditting can turn images of horses with white backgrounds into images of horses with dark backgrounds (horses are an unseen class).
NeuronEditting turns the horse darker as well. It seems that one could change the brightness and contrast of the image to obtain a similar effect, or one could or take the geometric mean of the image in [0,1] with the average target image and obtain a similar effect. Such traditional methods are also robust to unseen classes. Moreover, NeuronEditting's ability to change the brightness of the image is not that surprising given that brightness is some of the most basic image information. (In point of fact it is captured by the DC Coefficient, the very first coefficient from the discrete cosine transform which is used in JPEG.) What else can NeuronEditting do in the image domain? Can this be used to rotate or reflect MNIST digits? The biological experiments also appear to involve simple input transformations.
Fine points:
- "an edit function between the the"
- I am not sure the speculation about this method's loose relation to word2vec belongs in a scientific work. Both involve modifications to a neural representations, but no further relation is justified in the paper.
- Was the dataset partitioning for the CIFAR-10 experiment done manually? If not, what process partitioned the dataset?
Edit: Some of my suggestions were incorporated in the rebuttal, but my sentiment is still that this is almost at the acceptance threshold. The large focus on biology makes much of this paper harder to evaluate or appreciate.