Task: Language Identification
Input (Spanish, English): This work proposes to compute embeddings of symbolic expressions (e.g., boolean expressions, or polynomials) such that semantically equivalent expressions are near each other in the embedded space.  The proposed model uses recursive neural networks where the architecture is made to match that of the parse tree of a given symbolic expression.  To train the model parameters, the authors create a dataset of expressions where semantic equivalence relationships are known and minimize a loss function so that equivalent expressions are closer to each other than non-equivalent expressions via a max-margin loss function.  The authors also use a “subexpression forcing” mechanism which, if I understand it correctly, encourages the embeddings to respect some kind of compositionality.  Results are shown on a few symbolic expression datasets created by the authors and the proposed method is demonstrated to outperform baselines pretty convincingly.  I especially like the PCA visualization where the action of negating an expression is shown to correspond roughly to negating the embedding in its vector space — it is a lot like the man - woman + queen = king type embeddings that we see in the word2vec and glove style papers.    The weakest part of the paper is probably that the setting seems somewhat contrived — I can’t really think of a real setting where it is easy to have a training set of known semantic equivalences, but still more worth it to use a neural network to do predictions.   The authors have also punted on dealing with variable names, assuming that distinct variables refer to different entities in the domain.  This is understandable, as variable names add a whole new layer of complexity on an already difficult problem, but also seems high limiting.  For example, the proposed methods would not be useable in an “equation search engine” unless we were able to accurately canonicalize variable names in some way.  Other miscellaneous points: * Regarding problem hardness, I believe that the problem of determining if two expressions are equivalent is actually undecidable — see the “word problem for Thue systems”.  Related to this, I was not able to figure out how the authors determine ground truth equivalence in their training sets.  They say that expressions are simplified into a canonical form and grouped, but this seems to not be possible in general, so one question is — is it possible that equivalent expressions in the training data would have been mapped to different canonical forms?  Would it have been easier/possible to construct and compare truth tables? * The “COMBINE” operation uses what the authors describe as a residual-like connection.  Looking at the equations, the reason why this is not actually a residual connection is because of the weight matrix that is multiplied by the lower level l_0 features.  A true residual connection would have passed the features through unchanged (identity connection) and would have also been better at fighting gradient explosion…. so is there a reason why this was used rather than an identity connection? * In table 3, the first tf-idf entry: a + (c+a) * c seems equivalent to a + (c * (a+c)) * Vertical spacing between Figure 4 caption and body of text is very small and looks like the caption continues into the body of the text.
Output (English): en