The authors develop a way learn subspaces of multiple views such that data point neighborhoods are similar in all of the views.  This similarity is measured between distributions of neighbors in pairs of views. The motivation is that this is a natural criterion for information retrieval.  I like the idea of preserving neighborhood relationships across views for retrieval tasks. And it is nice that the learned spaces can have different dimensionalities for different views.  However, the empirical validation seems preliminary.  The paper has been revised from the authors' ICLR 2016 submission, and the revisions are welcome, but I think the paper still needs more work in order to be publishable.  In its current form it could be a good match for the workshop track.  The experiments are all on very small data sets (e.g. 2000 examples in each of train/test on the MNIST task) and not on real tasks.  The authors point out that they are not focusing on efficiency, and presumably computation requirements keep them from considering larger data sets.  However, it is not clear that there is any conclusion that can be drawn that would apply to more realistic data sets.  Considering the wealth of work that's been done on multi-view subspace learning, with application to real tasks, it is very hard to see this as a contribution without showing that it is applicable in such realistic settings.  On a more minor point, the authors claim that no other information retrieval based approaches exist, and I think this is a bit overstated.  For example, the contrastive loss of Hermann & Blunsom "Multilingual models for compositional distributed semantics" ACL 2014 is related to information retrieval and would be a natural one to compare against.  The presentation is a bit sloppy, with a number of vague points and confusing wordings.  Examples: - the term "dependency" gets used in the paper a lot in a rather colloquial way.  This gets confusing at times since it is used in a technical context but not using its technical definition. - "an information retrieval task of the analyst": vague and not quite grammatical - "the probability that an analyst who inspected item i will next pick j for inspection" is not well-defined - In the discussion of KL divergence, I do not quite follow the reasoning about its relationship to the "cost of misses" etc.  It would help to make this more precise (or perhaps drop it?  KL divergence is pretty well motivated here anyway). - Does C_{Penalty} (7) get added to C (6), or is it used instead?  I was a bit confused here. - It is stated that CCA "iteratively finds component pairs".  Note that while CCA can be defined as an iterative operation, it need not (and typically is not) solved that way, but rather all projections are found at once. - How is PCA done "between X_i^1 and X_i^2"? - "We apply nonlinear dimensionality algorithm": what is this algorithm? - I do not quite follow what the task is in the case of the image patches and stock prices.  Other minor comments, typos, etc.: - The figure fonts are too small. - "difference measures" --> "different measures" - "...since, hence any two...": not grammatical - "between feature-based views and views external neighborhoods": ?
Output: en