Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: The authors frame the decomposition of the Laplacian equation as an unsupervised regression problem that is using a 5-level (and fully connected?) neural network as regression function.  A cost function to be used in the optimization is proposed that is expanded to eigendecomposition problems of increasing complexity. A comparison with a classical method indicates that the proposed approach is comparable to or better than the former for the given task.

I like the overall approach and the idea of framing the solution of the eigenvalue decomposition as an ANN regression problem, and I would see that this is of interest to ICLR. From an application perspective, the authors refer to computer vision tasks, I would be interested in a somewhat deeper discussion of constraints of the approach, for example:

As both methods are somewhat on par in terms of various performance indicators I was wondering whether the proposed approach is faster?

In real world problem critical information is contained in the boundary conditions. Could the authors discuss pros and cons of their approach with respect to them? And, again, compare to existing methods?



review_title: Solving for eigenvalues via unsupervised ANN regression 

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper proposes a framework to estimate the eigenfunctions and eigenvalues of (non-linear) differential operators. The main idea is to parameterize the eigenfunction with a neural network and minimize a loss function which enforces the definition of eigenfunctions along with auxiliary terms enforcing boundary conditions, smoothness and getting rid of de-generate solutions. 

My main concern is the lack of novelty/understanding what the contribution is. Going through the existing literature (section 2.1 in related work), it seems that the standard way to solve a differential equation using deep networks is to parameterize the solution using a deep network and optimize a loss which captures the physical relation (that is the PDE itself) along with auxiliary terms for smoothness, boundary condition etc. In that sense, it is not clear to me what the contribution of the paper is. The main loss in the paper is definition of eigenfunction since that is the problem they're interested in and in the literature for solving PDEs, the main loss is the definition of the PDE itself. The authors should put their contribution in context with existing literature and highlight the changes they've made. 

In addition, it's unclear to me why we are particularly interested in the lowest M-eigenpairs. Is there a reason to concentrate on the lowest eigenvalues instead of the highest? I don't think this is motivated in the paper. Further, section 7 says " Furthermore, the inverse power method finds only a single eigenpair, while the proposed method outputs the M smallest eigenpairs.". Can we not find the smallest pair and run the method again with a constraint that the eigenfunction should be orthogonal to the one we already found to find the next smallest eigenfunction? (similar to what we do in power method?)

Finally, the experiments only concentrate on a Laplacian operator. In order to claim that is general and works across multiple operators, I believe that the results should be demonstrated  on multiple differential operators (of varying complexity levels). Unfortunately, I do not know enough about differential operators to suggest what these test cases should be. 

review_title: Official Review

final_score: rating: 4: Ok but not good enough - rejection
review_content: The manuscript proposes a deep learning solver for the eigenvalue problem of differential self-adjoint operators. Specifically, the aim is to calculate M lowest eigenvalues and their corresponding eigenfunctions. This work is a natural follow up of the work by Bar and Sochen (2019) for solving PDE-based problems.  The unsupervised loss resembles the loss suggested in Bar and Sochen (2019), with addition of mainly two terms: (a) The Rayleigh Quotient term and (b) the orthogonality constraint.  In practice, the proposed framework is tested on the 1D and 2D Laplace operators on regular domains. 

Pros. 
1.	The authors address an important and practical problem 
2.	The manuscript is self-contained, written very well and easy to follow 
3.	The proposed framework has the potential to handle eigenvalue problem of self-adjoint differential operators over irregular domains. 
4.	Mesh-free approach


Cons. 

1.	Incremental novelty. I feel that that the proposed framework has a quite limited novelty as it naturally extends the work by Bar and Sochen (2019)
2.	Generalization.  Practically, the proposed framework is an optimization framework and not a training framework. In other words, each new instance requires re-training of the neural net, in case of modification of the coefficients of the differential operator  and /or the boundary conditions.  Moreover, in the 2D case, M networks are optimized simultaneously.  
3.	I think that  it is worthwhile to refer to existing works, solving the eigenvalue problem by neural net.
4.	Limited experimental work.  The proposed framework has the potential to handle irregular domains. However, as far as I can see the experiments are done over regular domains for a special instance of differential operator, namely the Laplace operator.
5.	Comparison.   The comparison is made versus the inverse power method. I think that a comparison to a very relevant solver, LOBPCG, is missing.


review_title: I vote for reject.

final_score: rating: 3: Clear rejection
review_content: ##########################################################################

Summary:

This paper proposes to use deep learning neural network to model eigenfunction while solving a generic eigenvalue problems. Since such problems is very generic this application of such framework are huge and the experiments show some promising results. 

##########################################################################

Reasons for score: 
 
Overall, I vote for rejecting (see cons for more details). The idea is very good, but many important information are missing. For example it is impossible to reproduce the results since we don't have any clue on the used networks.

##########################################################################

Pros:  

1. This paper proposes a new framework for generic eigenvalue problem which seems promising.

2. The framework is able to deal with several eigenfunction and almost assure the orthogonality of the function.

3. This paper provides comprehensive experiments, that shows that such methods cam catch complex eigenfunction. 

##########################################################################

Cons: 

1. The notation is confusing on some parts. First please use <,> for scalar product instead of (,) since you have functions with two arguments. Second I don't w_u is a good notation for the weights , for example in equation 6 we don't know which weights it is. Perhaps using a function which gives the associated weights could help.

2.  When dealing with multiple eigenfunction, there is en additional term is the equation 6 for the orthogonality constraints. However it is not clear on how it is approximate, I assume that it is something similar to equation (3). Please clarify.

3. Again on the orthogonality constraint, the complexity of such constraint is huge. It is impossible to deal with many eigenfunction. Such limitation can drastically reduce the interest of the framework, I think there should be some discussion on this point.

4. On the implementation details, we don't have any clue on which kind of neural network are used. Are they just dense network?

5. Again on the implementation details, the loss function of equation 6 is highly non-convex and I assume that there is several way to try solving it. I don't think it is possible to deal with all the networks (the u) at once, so I figure there is a solving scheme (most likely alternate minimization) behind. Please clarify as it may answer to one of my previous remarks. 

##########################################################################

Questions during rebuttal period: 
 
Please address and clarify the cons above 

#########################################################################

Some typos: 

all pages: be careful with the citation, when citing a paper please add parenthesis

page 2: There is an extra parenthesis in equation 2
review_title: Lack important information: Reject

Meta-review: The paper addresses the problem of solving for the eigenpairs of a self-adjoint differential operator. This problem, of course, is classical; the main innovations here are 
a) the use a parametric form of the (pointwise) solution using a (shallow) neural network so as to avoid discretization, and 
b) obtaining multiple eigenpairs simultaneously as outputs. 
The techniques themselves (i.e., the architecture, the loss function, and the training procedure) are fairly standard.

There was some variance in the review scores. The reviewers appreciated the importance of the problem and the direction adopted by the authors. However, concerns were raised regarding: the limitations of the experimental evaluation; a lack of sufficient distinction from prior work; *very* limited comparisons with prior approaches; and the limited demonstration of generalizability. I agree with all these criticisms, and therefore vote to reject. 

Decision: Reject