From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Reject
metareview:_summary,_strengths_and_weaknesses: This paper proposes a machine learning inspired heuristic to adaptively choose an important parameter of ADMM when solvin quadratic convex programs. The proposed method added a temporal component via a gated recurrent unit, which allows the model to incorporate consecutive information from previous iterations.
In addition, the authors proposed heterogeneous graph attention for the embedding.
The experiments show some improvements in some situations.

justification_for_why_not_higher_score: Using spatial and temporal with RL is not innovative. 
The intuition of using the graph attention layer is only motivated by giving the definition of the contextual MDP.
Concerning experiments, the proposed method is only compared with QP ADMM methods by setting matrices to be zeros on some random data. It could be more convincing to compare with ADMM algorithms designed for LP on some real datasets.
justification_for_why_not_lower_score: It empirically show how attention and temporal information can improve NN based parameter selection heuristics.
The experiments are thorough, especially after wall clock time has been added.
The ablation study is of interest for ascertaining the influence of the parameters.

summary_of_AC-reviewer_meeting: No meeting: the reviewers never answered my emails.

From: ICLR.cc/2023/Conference/Paper5664/Reviewer_1ZMW
confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper considers the problem of QP in which the authors propose an adaptive scheme for ADMM step size selection (CA-ADMM). CA-ADMM obtains the step sizes using an MDP. The context for the MDP is arrived at using spatial and temporal means, wherein they use a graph neural network defined over the ADMM variables to generate the former and use the previous history to generate the latter. They do comprehensive experiments on various datasets illustrating that the proposed method leads to lesser number of admm iterations than the compared baselines. 
strength_and_weaknesses: Strengths
- Novelty of the proposed architecture for ADMM updates
- The experimentatal results are strong. 
-- The proposed method performs convincingly better than the baselines.
-- The dependency on the problem size is also studied in the experiments. 
-- The ability to transfer trained model from smaller sizes to larger settings.


Questions
- (Minor) Table 2. What do the rows with labels "transferred" and "in-domain" refer? Some clarification may help understanding this better. 
- Was there any experimentation done comparing how frequently \rho should be updated ? Or is there any objective justification for the update once in 10 iterations. 
- How might the performances when we compare well-conditioned vs ill-conditioned matrix P?


clarity,_quality,_novelty_and_reproducibility: The paper is written well and easy to follow. The presented results seem novel and significant. 
summary_of_the_review: The paper proposes a novel scheme for ADMM weights using an MDP based updates. The novelty of the scheme and the experimental results are convincing. 
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 8: accept, good paper

From: ICLR.cc/2023/Conference/Paper5664/Reviewer_L3nw
confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
summary_of_the_paper: This paper proposes a context-aware adaptive mechanism to adjust the step-size parameter rho in ADMM for solving convex quadratic programming problems, denote as CA-ADMM. It extracts the spatio-temporal context during the ADMM iterations. Numerical experiments on various type of QP sets are reported.
strength_and_weaknesses: Strength:
i) The proposed method seems to have a substantial improvement over RLQP and OSQP in terms of number of iterations

Weakness:
i) The reported experimental results are on number of iterations. It would be much more meaningful to report the CPU time as well. In particular, the time for finding a good rho in CA-ADMM is supposed to take more time than that in OSQP. It is important to note if the decrease in number of iterations can compensate the overhead in finding a good rho.
ii) The paper is more on the empirical side. Therefore, it is very important to see the real performance of the proposed method. However, all the experiments are done on synthetic data sets (randomly generated). It is necessary to compare the proposed method with OSQP on public available real QP data set, for example, the Meszaros' QP set (http://old.sztaki.hu/~meszaros/public_ftp/qpdata/). 
iii) If the paper only considers convex quadratic programming, please emphasize this in the title and abstract. There is no experiments reported on non-convex quadratic programming problems.

clarity,_quality,_novelty_and_reproducibility: Clarity: the paper is well written and easy to read
Novelty: the proposed method is new with fair novelty

summary_of_the_review: The paper proposes a new method for using ML techniques to accelerate ADMM for convex quadratic programming. The merit of this paper is more from the practical point of view. However, the reported experimental result has two main issues. First, the experiment is only conducted on synthetic/randomly generated data set. Second, only number of iterations is reported for comparing the efficiency between algorithms. 
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper5664/Reviewer_nZJ9
confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
summary_of_the_paper: This paper introduces a machine learning based heuristic method to adaptively choose an important parameter of ADMM when applying to solve quadratic programming problems. The proposed method added a temporal component via a gated recurrent unit, which allows the model to incorporate consecutive information from previous iterations. In addition, the authors proposed heterogeneous graph attention for the embedding. The experiments show some improvements.
strength_and_weaknesses: Strengths:
1. This paper tests out a clear hypothesis that attention and temporal information can improve NN based parameter selection heuristics, and provide an empirical evaluation of their new approach.
2. The experiments are thorough and help ascertain the importance of each new contribution independently. The ablation study also does a reasonably good job of including necessary experiments for ascertaining the importance of different parameters and other aspects needed for reproducibility.

Weakness:
1. The technical contribution seems relatively incremental since both GRU and attention are commonly used. In addition, the intuition why GRU and attention can improve performance is not clear.
2. The authors only showed improvement in the iteration. How about the total time? In my opinion, using a graph neural network to get the parameter $\rho$ may increase the time complexity and resource consumption.
3. Why doesn't the author also apply similar algorithms for using ADMM to solve linear programming problems? If the idea works well for QP, it should work on LP as well. If not, providing reasons why it doesn't work on LP may help the community to understand the limitations and strengths of the proposed method.

clarity,_quality,_novelty_and_reproducibility: The paper is well-written and easy to follow.
summary_of_the_review: Overall, this paper has some non-trivial contributions but not significant ones. Thus it needs to be significantly improved to achieve the standard of ICLR.

=======After rebuttal=====

Thanks for answering my questions.

After reading your rebuttal, I still think using spatial and temporal with RL is not innovative. In addition,  only given the definition of contextual MDP is not enough to support the intuition of using the graph attention layer. Therefore, I still believe the contribution of this work is limited.

For the new experiments, it is only compared with QP ADMM methods by setting matrices to be zeros on some random data. I think it will be more convincing to compare with ADMM algorithms designed for LP on some real datasets.

Thus, I would like to maintain my original score.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold