Task: Language Identification
Input (Spanish, English): A new sparse coding model is introduced that learns features jointly with their transformations. It is found that inference over per-image transformation variables is hard, so the authors suggest tying these variables across all data points, turning them into global parameters, and using multiple transformations for each feature. Furthermore, it is suggested to use a tree of transformations, where each path down the tree generates a feature by multiplying the root feature by the transformations associated with the edges. The one-layer tree model achieves similar reconstruction error as traditional sparse coding, while using fewer parameters.  This is a nice addition to the literature on sparse coding and the literature on learning transformation models. The authors identify and deal with a difficult inference problem that can occur in transformation models. That said, I am skeptical about the usefulness of the general approach. The authors take it as a given that “learning sparse features and transformations jointly” is an important goal in itself, but this is never really argued or demonstrated with experiments. It doesn’t seem like this method enables new applications, extends our understanding of learning what/where pathways in the brain, or improve our ability to model natural images.  The authors claim that the model extracts pose information, but although the model explicitly captures the transformation that relates different features in a tree, at test time, inference is only performed over the (sparse) coefficient associated with each (feature, transformation) combination, just like in sparse coding. It is not clear what we gain by knowing that each coefficient is associated with a transformation, especially since there are many models that do this general “what / where” split.  It would be good to check that the x_{v->b} actually change significantly from their initialization values. The loss surface still looks pretty bad even for tied transformations, so they may actually not move much. Does the proposed model work better according to some measure, compared to a model where x_{v->b} are fixed and chosen from some reasonable range of parameter values (either randomly or spaced evenly)?  One of the conceptually interesting aspects of the paper is the idea of a tree of transformations, but the advantage of deeper trees is never demonstrated convincingly. It looks like the authors have only just gotten this approach to work on toy data with vertical and horizontal bars.  Finally, it is not clear how the method could be extended to have multiple layers. The transformation operators T can be defined in the first layer because they act on the input space, but the same cannot be done in the learned feature space. It is also not clear how the pose information should be further processed in a hierarchical manner, or how learning in a deep version should work.  In summary, I do not recommend this paper for publication, because it is not clear what problem is being solved, the method is only moderately novel and the novel aspects are not convincingly shown to be beneficial.
Output (English): en