Task: Language Identification
Input (Spanish, English): Variational auto-encoders, adversarial networks, and kernel scoring rules like MMD have recently gained popularity as methods for learning directed generative models and for other applications like domain adaptation. This paper gives an additional method along the scoring rules direction that uses the matching of central moments to match two probability distributions. The technique is simple, and in the case of domain adaptation, highly effective.  CMD seems like a very nice and straightforward solution to the domain adaptation problem. The method is computationally straightforward to implement, and seems quite stable with respect to the tuning parameters when compared to MMD. I was skeptical reading through this, especially given the fact that you only use K=5 in your experiments, but the results seem quite good. The natural question that I have now is: how will this method do in training generative models? This is beyond the scope of this paper, but it’s the lowest hanging fruit.  Below I give more detailed feedback.  One way to speed up MMD is to use a random Fourier basis as was done in “Fastmmd: Ensemble of circular discrepancy for efficient two-sample test” by Zhao and Meng, 2015. There are also linear time estimators, e.g., in “A Kernel Two-Sample Test“ by Gretton et al., 2012. I don’t think you need to compare against these approaches since you compare to the full MMD, but they should be cited.  The paper “Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy” by Sutherland et al. submitted to ICLR 2017 as well, discusses techniques for optimizing the kernel used in MMD and is worth citing in section 3.  How limiting is the assumption that the distribution has independent marginals?  The sample complexity of MMD depends heavily on the dimensionality of the input space - do you have any intuitions about the sample complexity of CMD? It seems like it's relatively insensitive based on the results in Figure 4, but I would be surprised if this were the case with 10,000 hidden units. I mainly ask this because with generative models, the output space can be quite high-dimensional.  I’m concerned that the central moments won’t be numerically stable at higher orders when backpropagating. This doesn’t seem to be a problem in the experimental results, but perhaps the authors could comment a bit on this? I’m referring to the fact that ck(X) can be very large for k >= 3. Proposition 1 alleviates my concerns that the overall objective is unstable, I’m referring specifically to the individual terms within.  Figure 3 is rather cluttered, and aside from the mouse class it’s not clear to me from the visualization that the CMD regularizer is actually helping. It would be useful to remove some of the classes for the purpose of visualization.  I would like some clarification about the natural geometric interpretations of K=5. Do you mean that the moments up to K=5 have been well-studied? Do you have any references for this? Why does K >= 6 not have a natural geometric interpretation?  Figure 4 should have a legend
Output (English): en