Venue: ICLR 2018 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes a method of learning sparse dictionary learning by introducing new types of priors. Specifically, they designed a novel idea of defining a metric to measure discriminative properties along with the quality of presentations.
It is also presented the power of the proposed method in comparison with the existing methods in the literature.

Overall, the paper deals with an important issue in dictionary learning and proposes a novel idea of utilizing a set of priors. 

To this reviewer’s understanding, the thresholding parameter $\tau_{c}$ is specific for a class $c$ only, thus different classes have different $\tau$ vectors. If so, Eq. (6) for approximation of the measure $D(\cdot)$ is not clear how the similarity measure between ${\bf y}_{c,k}$ and ${\bf y}_{c1,k1}$, \ie, $\left\|{\bf y}_{c,k}^{+}\odot{\bf y}_{c1,k1}^{+}\right\|_{1}+\left\|{\bf y}_{c,k}^{+}\odot{\bf y}_{c1,k1}^{+}\right\|_{1}$ and $\left\|{\bf y}_{c,k}\odot{\bf y}_{c1,k1}\right\|_{2}^{2}$, works to approximate it. It would be appreciated to give more detailed description on it and geometric illustration, if possible.

There are many typos and grammatical errors, which distract from reading and understanding the manuscript.
review_title: Long paper with good proofs

final_score: rating: 4: Ok but not good enough - rejection
review_content: Summary:
The paper proposes a model to estimate a non-linear transform of data with labels, trying to increase the discriminative power of the transformation while preserving information. 

Quality:
The quality is potentially good but I misunderstood too many things (see below) to emit a confident judgement.

Clarity:
Clarity is poor. The paper is overall difficult to follow (at least for me) for different reasons. First, with 17 pages + references + appendix, the length of the paper is way above the « strongly suggested limit of 8 pages ». Second, there are a number of small typos and the citations are not well formatted (use \citep instead of \citet). Third, and more importantly, several concepts are only vaguely formulated (or wrong), and I must say I certainly misunderstood some parts of the manuscript.
A few examples of things that should be clarified:
- p4: « per class c there exists an unknown nonlinear function […] that separates the data samples from different classes in the transformed domain ». What does « separate » mean here? There exists as many functions as there are classes, do you mean that each function separates all classes? Or that when you apply each class function to the elements of its own class, you get a separation between the classes? (which would not be very useful)
- p5: what do you mean exactly by « non-linear thresholding function »?
- p5: « The goal of learning a nonlinear transform (2) is to estimate… »: not sure what you mean by « accurate approximation » in this sentence.
- p5 equation 3: if I understand correctly, you not only want to estimate a single vector of thresholds for all classes, but also want it to be constant. Is there a reason for the second constraint?
- p5 After equation 4, you define different « priors ». The one on z_k seems to be Gaussian; however in equation 4, z_k seems to be the difference between the input and the output of the nonlinear threshold operator. If this is correct, then the nonlinear threshold operator is just the addition of a Gaussian random noise, which is really not a thresholding operator. I suppose I misunderstood something here, some clarification is probably needed.
- p5 before equation 5, you define a conditional distribution of \tau_c given y_{c,k} ; however if you define a prior on \tau_c given each point (i.e., each $k$), how do you define the law of \tau_c given all points?
- p5 Equation 6: I dont understand what the word « approximation » refers to, and how equation 6 is derived. 
-p6 equation 7 : missing exponent in the Gaussian distribution
- p6-7: to combine equations 7-8-9 and obtain 10, I suppose in equation 7 the distributions should be conditioned on A (at least the first one), and in equation 9 I suppose the second line should be removed and the third line should be conditioned on A; otherwise more explanations are needed.
- Lemma 1 is just unreadable. Please at least split equations in several lines.

Originality:
As far as I can tell, the approach is quite original and the results proved are new.

Significance:
The method provides a new way to learn discriminative features; as such it is a variant of several existing methods, which could have some impact if clarity is improved and a public code is provided.
review_title: A potentially interesting idea, but the paper is a bit hard to follow

final_score: rating: 5: Marginally below acceptance threshold
review_content: Overview:
This paper proposes a method for learning representations using a “non-linear transform”. Specifically, the approach is based on the form: Y =~ AX, where X is the original data, A is a projection matrix, and Y is the resulting representation. Using some assumptions, and priors/regularizers on Y and A, a joint objective is derived (eq. 10), and an alternating optimization algorithm is proposed (eq. 11 and 14). Both objective and algorithm use approximations due to hardness of the problem. Theoretical and empirical results on the quality and properties of the representation are presented.
Disclaimer: this is somewhat outside my area of expertise, so this is a rather high-level review. I have not thoroughly checked proofs and claims.

Comments:
-I found the presentation quality to be rather poor, making it hard to fully understand and evaluate the approach. In particular, the motivation and approach are not clear (sec. 1.2), making it hard to understand the proposed method. There is no explicit formulation, instead there are references to other models (e.g., sparsifying transform model) and illustrative figures (fig. 1 and 2). Those are useful following a formal definition, but cannot replace it. The separation between positive and negative elements of the representation is not motivated and explained in a footnote although it seems central to the proposed approach.
- The paper is 17 pages long (24 pages with the appendix), so I had to skim through some parts. Due to the extensive scope, perhaps a journal submission would be more appropriate.

Minors:
- Vu & Monga 2016b and 2016c are the same.
- p. 1: meaner => manner
- p. 1: refereed => referred
- p. 1: “a structural constraints”; p. 2: “a low rank constraints”, “a pairwise constraints”; p. 4: “a similarity concentrations”, “a numerical experiments”, and others...
- p. 2, 7: therms => terms
- p. 2: y_{c_1,k_2} => y_{c_1,k_1}?
- p. 3, 4: “a the”
- p. 5: “an parametric”
- p. 8: ether => either
- Other typos… the paper needs proofreading.

review_title: A paper on representation learning with non-linear transform. Presentation quality should be improved.

Meta-review: This paper proposes an approach for learning a sparsifying transform via a set of nonlinear transforms at learning time.  The presentation needs a lot of work.  The original paper was 17 pages long and very difficult to understand.  The revised paper is 12 pages long, which is still too long for the content.  The paper needs to better distinguish between the major and minor points.  It is still too difficult to judge the contribution.
Decision: Reject