Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes to use implicit differentiation in value iteration networks, allowing for a deeper network structure/more iterations of planning. This is because the implicit gradient has a constant complexity with respect to the number of planning iterations.
strength_and_weaknesses: **strengths**

* The paper identifies and fixes an issue with an existing method. Value iteration networks do not scale to high numbers of planning iterations. Implicit differentiation reduces the cost of the gradient computation, which allows using many more planning steps.
* Experiments show that implicit gradients allow using more planning iterations for the same cost of gradient computation. The authors also show that successful models can be trained with very high numbers of planning iterations using implicit gradients, while the same is not true of explicit ones.

**weaknesses**

I have some concerns about the results and the presentation. I will make an overview of the experiments and how I interpret the results below to make that clear.

**overview of experiments**

The paper has 4 experiment setups: 1) 2D navigation 2) visual navigation 3) configuration-space manipulation 4) work-space manipulation.

**2D navigation**

There are maps of size 15x15, 27x27 and 49x49. According to Figure 2, in both 15x15 and 27x27 there exist explicit planners which reach 100% success. In 49x49 maps, the best explicit planner (SymVIN with 30 iters) reaches about 85% success.

According to Figure 4, the implicit planners match the performance of explicit ones in all scenarios, and even exceed that in 49x49 with 80 iterations using ConvGPPN. That being said, according to Figure 6 the runtimes of implicit planners in the 49x49 task seem impractical for all models (except VIN, which performs poorly).

To sum up, implicit vs explicit perform roughly on par. There are setups where implicit works better, but the runtime gets quite high.

**visual navigation**

According to table 1, implicit methods clearly outperform explicit ones. However, table 1 averages results across 30, 50 and 80 iterations. We can find individual runs in Figure 13. Here, the best explicit methods are SymVIN and ConvGPPN with 30 iterations, which reach ~100% success. The implicit ones also reach this performance. To sum up, implicit vs explicit perform on par.

**configuration-space manipulation**

There are 18x18 and 36x36 setups. Again, table 1 shows implicit methods outperforming the others on averaged results. Figure 11 contains individual results for 30, 50 and 80 iterations. In both 18x18 and 36x36, the best explicit method (SymVIN with 30 iters) reaches ~100% success rate. Implicit methods also reach this performance. So again, the two are on par.

**work-space manipulation**

Similar to configuration space, averaged results favor implicit methods. According to Figure 11, the best explicit methods are SymVIN and ConvGPPN with 30 iterations, and they reach between 85 and 90% success. The best implicit method seems to be SymVIN with 50 iterations. This performs similar to the explicit ones, but perhaps with a slight edge. Without exact numbers I'd have to say the best explicit and the best implicit perform on par.

---

Looking at these results, I have two main concerns:

* Implicit and explicit methods appear to perform on par, if we compare the best explicit method for each setup with the best implicit method. The only setup where implicit has an edge is the 49x49 2D navigation task. However the runtimes of the implicit method here appear impractical. The backward pass of SymVIN takes 50 seconds according to Figure 6, which is quite long. That would mean over an hour for 100 gradient updates.
* I find the presentation a bit misleading at times. In Table 1, it doesn't make sense to me to average across planning iterations. A fair comparison would be best-vs-best. Also, I find that there is not enough discussion of runtimes. I greatly appreciate Figure 6 but I still think more space should be dedicated to discussing runtimes. The results don't seem to be as simple as "implicit gradient is faster". There are some settings where the implicit gradient is slower than the explicit one (e.g. 49x49 SymVIN with 30 iterations) and settings where the fact that the implicit gradient is faster doesn't matter too much because it is still not fast enough (e.g. 49x49 ID-SymVIN backward pass takes close to a minute). A fair comparison would also include the runtime of the best implicit method against that of the best explicit one.

Please let me know if I'm misinterpreting the results here.

My recommendation would be to point out a setup where a) the implicit version is feasible, while the explicit is not b) the implicit version has an edge over the explicit one. By b) I mean that the longer planning horizon allows the method to reach a level of performance that an explicit gradient cannot reach with any number of iterations that is applicable or any choice of model. This would make it easier for the reader to understand when implicit gradients are helpful. I have searched for such a setup in the paper but couldn't find it (see my process above under "overview of experiments").

**some minor points**

* Can the authors define what they mean by divergence? Do you mean that the gradient vanishes or explodes?
* One missing related work that I spotted: [1]. It has a similar goal as this paper: scaling up planning to larger environments.

[1] Value Iteration Networks on Multiple Levels of Abstraction, http://www.roboticsproceedings.org/rss15/p14.pdf
clarity,_quality,_novelty_and_reproducibility: The paper is well-written. Source code is not available (though a release is planned). There is an appendix with ample implementation details. I have some issue with the presentation and the results, which I detailed under "weaknesses".
summary_of_the_review: The paper proposes using implicit differentiation in value iteration networks (VINs). This allows using many more planning iterations than usual. The authors show that they can successfully train VINs with very high numbers of planning iterations, which is not possible using explicit gradients. I generally do not see a performance gap between implicit and explicit gradients in the experiments. The few cases where there is a benefit for implicit methods seem to also come with a very high runtime. The presentation does not make this very clear due to the reporting of averaged results. I find the idea of using implicit differentation in VINs very promising and interesting, but the experimental results and presentation currently do not convince me. The paper can be made a lot stronger by including experiments which show implicit methods clearly outperforming explicit ones and with reasonable runtimes. At the current stage, I see the paper as below the threshold of acceptance.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper applied the implicit differentiation to the equilibrium state of the VIN-based path planning pipelines. Since the forward computation of VIN is effectively solving a fixed point of the Bellman equation, it is natural to treat this process as a deep equilibrium model (DEQ Bai et al. 2019), which can backpropagate the gradients without storing the entire forward computational graph. 
strength_and_weaknesses: Strengths

DEQ is suitable with the VIN-based path planning framework. The forward iteration of VIN can be viewed as solving a fixed-point equation, which is exactly what DEQ should deal with. The back-propagation of DEQ only relies on the fixed point itself, so independent of the forward solving. As a result, we can have a longer forward iteration to solve for a better convergence in some larger-scale cases.

Weaknesses

Although this paper chooses an appropriate model to tackle the VIN-based path finding problems, the technical novelty itself is somehow limited. Both the forward network structures and backward implicit differentiation schemes are from existing models. 

Most importantly, the experiments cannot fully back up this paper’s main claim, that this method can scale up the path planning algorithms. The biggest map size is only 49x49, which is even less than the 100x100 examples used in the baseline SymVIN [1]. To validate the advantages of implicit differentiation, a bigger map size (with which previous baselines cannot deal) might be needed.

[1] Integrating Symmetry into Differentiable Planning 


clarity,_quality,_novelty_and_reproducibility: This paper is clear and easy to follow. Experiments can serve as proof of concept but lack larger-scale tests. The authors provide their code. The building blocks of this pipeline are simple and should be reproducible. 

summary_of_the_review: In summary, this paper could be a good application of DEQ. However, the technical novelties are limited. Most of the components are from existing work. The experiments are on small scales and not compelling enough to back up the claimed scalability.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes to scale differentiable planning such as Value Iteration Networks (VINs) with implicit differentiation. Section 3 summarizes value iteration networks, symmetric VINs, and gated path planning networks and how the fixed point of the value iteration can be seen as an implicit function, i.e. Bellman optimality provides an implicit function. Section 4.1 goes on to review how to compute the implicit derivatives or approximations to them, and Section 4.2 instantiates the full pipeline for implicit planning.
strength_and_weaknesses: Strengths
+ The idea of implicitly differentiating planners that compute a fixed point is appealing
+ The implicit planners on the 2d navigation tasks in Figures 4, 6 and 7 and Table 1 convincingly improve upon the performance in contrast to the corresponding planners trained without implicit differentiation

Weaknesses
+ Sometimes the value iterations may take a long time to reach a fixed point. Is it hard in practice to ensure that a fixed-point is reaches so that the implicit derivatives remain stable?
clarity,_quality,_novelty_and_reproducibility: The paper is clear, well-written, and reproducible.
summary_of_the_review: I recommend to accept this paper as it combines a reasonable core contribution of implicitly differentiating planners along with extensive experimental demonstrations of how it enables them to scale.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper improves value-iteration networks by using implicit differentiation, thus allowing for scaling of these methods.  That contribution is nice, albeit that the merit of this approach would be stronger with wider experimentation (but that's true for any paper).


justification_for_why_not_higher_score: See meta-review.
justification_for_why_not_lower_score: See meta-review.
Decision: Accept: poster