Venue: ICLR 2017 workshop

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: I previously reviewed the paper and am attaching my review below.

I still have concern regarding the incorrect arguments raised in the paper (such as the nature of matching the source/target domain distributions), and it seems that the authors simply reorganized the sections down to the appendix section. As a result I am keeping my original recommendation, but will not argue if the AC decides to accept the paper.

*** Original review ***
Overall I think this is an interesting paper which shows empirical performance improvement over baselines. However, my main concern with the paper is regarding its technical depth, as the gist of the paper can be summarized as the following: instead of keeping the batch norm mean and bias estimation over the whole model, estimate them on a per-domain basis. I am not sure if this is novel, as this is a natural extension of the original batch normalization paper. Overall I think this paper is more fit as a short workshop presentation rather than a full conference paper.

Detailed comments:

Section 3.1: I respectfully disagree that the core idea of BN is to align the distribution of training data. It does this as a side effect, but the major purpose of BN is to properly control the scale of the gradient so we can train very deep models without the problem of vanishing gradients. It is plausible that intermediate features from different datasets naturally show as different groups in a t-SNE embedding. This is not the particular feature of batch normalization: visualizing a set of intermediate features with AlexNet and one gets the same results. So the premise in section 3.1 is not accurate.

Section 3.3: I have the same concern as the other reviewer. It seems to be quite detatched from the general idea of AdaBN. Equation 2 presents an obvious argument that the combined BN-fully_connected layer forms a linear transform, which is true in the original BN case and in this case as well. I do not think it adds much theoretical depth to the paper. (In general the novelty of this paper seems low)

Experiments:

- section 4.3.1 is not an accurate measure of the "effectiveness" of the proposed method, but a verification of a simple fact: previously, we normalize the source domain features into a Gaussian distribution. the proposed method is explicitly normalizing the target domain features into the same Gaussian distribution as well. So, it is obvious that the KL divergence between these two distributions are closer - in fact, one is *explicitly* making them close. However, this does not directly correlate to the effectiveness of the final classification performance.

- section 4.3.2: the sensitivity analysis is a very interesting read, as it suggests that only a very few number of images are needed to account for the domain shift in the AdaBN parameter estimation. This seems to suggest that a single "whitening" operation is already good enough to offset the domain bias (in both cases shown, a single batch is sufficient to recover about 80% of the performance gain, although I cannot get data for even smaller number of examples from the figure). It would thus be useful to have a comparison between these approaches, and also a detailed analysis of the effect from each layer of the model - the current analysis seems a bit thin.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: (I previously reviewed this paper for the main conference.  I will copy most of my comments here, removing criticisms that have since been addressed.)


Pros:

The method is very simple and easy to understand and apply.

The experiments demonstrate that the method compares favorably with existing methods on standard domain adaptation tasks.

The analysis in section 5.3.3 shows that only a small number of target domain samples are needed for adaptation of the network.

Good results for remote sensing domain adaptation included in appendix.


Cons:

There is little novelty -- the method is arguably too simple to be called a “method.” Rather, it’s the most straightforward/intuitive approach when using a network with batch normalization for domain adaptation.  (The alternative -- using the BN statistics from the source domain for target domain examples -- is less natural, to me.)


Overall, there’s not much novelty here, but the paper includes sufficient experimentation and interesting analysis, and it’s hard to argue that simplicity is a bad thing when the method is clearly competitive with or outperforming prior work on the standard benchmarks (in a domain adaptation tradition that started with “Frustratingly Easy Domain Adaptation”).
review_title: simple but effective domain adaptation approach

Decision: Accept