Task: Language Identification
Input (Spanish, English): The authors propose and evaluate using SPN's to generate embeddings of input and output variables, and using MPN to decode output embeddings to output variables. The advantage of predicting label embeddings is to decouple dependencies in the predicted space. The authors show experimentally that using SPN based embeddings is better than those produced by RBM's.  This paper is fairly dense and a bit hard to read. After the discussion, the main contributions of the authors are:  1. They propose the scheme of learning SPN's over Y and then using MPN's to decode the output, or just SPNs to embed X. 2. They propose how to decode MPN's with partial data. 3. They perform some analysis of when their scheme will lead to perfect encoding/decodings. 4. They run many, many experiments comparing various ways of using their proposed method to make predictions on multi-label classification datasets.  My main concerns with this paper are as follows:  - The point of this paper is about using generative models for representation learning. In their experiments, the main task is discriminative; e.g. predict multiple Y from X. The only discriminative baseline is a L2 regularized logistic regression, which does not have any structure on the output; it'd be nice to see how a discriminative structured prediction method would do, such as CRF or belief propagation.   - The many experiments suggest that their encoder/decoder scheme is working better than the alternatives; can you please give more details on the relative computation complexity of each method?  - One thing I'm still having trouble understanding is *why* this method works better than MADE and the other alternatives. Is it learning a better model of the distribution of Y? Is it better at separating out correlations in the output into individual nodes?  Does it have larger representations?   - I think the experiments are overkill and if anything, they way they are presented detract from the paper. There's already far too many numbers and graphs presented to be easy to understand.  If I have to dig through hundreds of numbers to figure out if your claim is correct, the paper is not clear enough. And, I said this before in my comments, please do not refer to Q1, Q2, etc. -- these shortcuts let you make the paper more dense with fewer words but at the cost of readability.  I *think* I convinced myself that your method works...I would love to see a table that shows, for each condition: (A) a baseline X->Y, (B) one *average* result across datasets for your method, and (C) one *average* result from a reasonable best competitor method. Please show for both the exact match and hamming losses, as that will demonstrate the gap between independent linear prediction and structured prediction. That would still be plenty of numbers but would make it much easier for the reader to verify your claims and you can put everything else in the Appendix.  E.g. something like:  Input | Predicted Output | Decoder | Hamming | Exact Match ---- X | P(Y) | CRF | xx.xx | xx.xx   (this is your baseline) SPN E_X | P(Y) | n/a | xx.xx | xx.xx  X | SPN E_Y | MPN | xx.xx | xx.xx  (given X, predict E_Y, then decode it with an MPN)  Does a presentation like that make sense? It's just really hard and time-consuming for me as a reviewer to verify your results, the way you've laid them out currently.
Output (English): en