Paper:
In spite of their great success, traditional factorization algorithms typically do not support features (e.g., Matrix Factorization), or their complexity scales quadratically with the number of features (e.g, Factorization Machine). On the other hand, neural methods allow large feature sets, but are often designed for a specific application. We propose novel deep factorization methods that allow efficient and flexible feature representation. For example, we enable describing items with natural language with complexity linear to the vocabulary size-this enables prediction for unseen items and avoids the cold start problem. We show that our architecture can generalize some previously published single-purpose neural architectures. Our experiments suggest improved training times and accuracy compared to shallow methods.

Review:
This paper presents a method for matrix factorization using DNNs. The suggestion is to make the factorization machine (eqn 1) deep, by grouping the features meaningfully (eqn 5), extracting nonlinear features from original inputs (deep-in, eqn 8), and adding additional nonlinearity after computing pairwise interactions (deep-out, eqn 7). From the methodology point of view, such extensions are relatively straightforward. As an example, from the experimental results, it seems the grouping of features is done mostly with domain knowledge (e.g., months of year) and not learned automatically. The authors claim the proposed method can circumvent the cold-start problem, and presented some experimental results on recommendation systems with text features.
While the application problems look quite interesting, in my opinion, the paper needs to make the context and contribution clearer. In particular, there is a huge literature in collaborative filtering, and I believe there is by now sufficient work on collaborative filtering with input features (and possibly dealing with the cold-start problem). I think this paper does not connect very well with that literature. When reading it, at times I felt the main purpose of this paper is to solve the application problems presented in experimental results, instead of proposing a general framework. I suggest the authors to demonstrate their method on some well-known datasets (e.g., MovieLens, Netflix), to give the readers an idea if the proposed method is indeed advantageous over more classical methods, or if the success of this paper is mostly due to clever processing of text features using DNNs.
Some detailed comments:
1. eqn 4 does not indicate any rank-r factors. 
2. some statements do not seem straightforward/justified to me:  
    -- the paper uses the word "inference" several times without definition
    -- "if we were interested in interpreting the parameters, we could constrain w to be non-negative ... ". Is this easy to do, and can the authors demonstrate this in their experiments and show interpretable examples?
    -- "Note that if the dot product is replaced with a neural function, fast inference for cold-start ...". 
3. the experimental setup seems quite unusual to me: "since we only observe positive labels, for such tasks in the test set we sample a labels according to the label frequency". This seems very problematic if most of the entries are not observed. Why cannot you use the typical evaluation procedure for collaborative filtering, where you hide some known entries during model training, and evaluate on these entries during test?