Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors propose a reinforcement learning approach to road vectorization. In contrast with previous works, the task is modeled as generating a graph as a variable-length edge sequence. 

The pipeline consists of multiple stages: semantic segmentation from RGB images (this results in binary maps), a transformer-based autoregression model (results in a collection of road graphs) and the RL part- muZero adapted for road graph extraction. It achieves competitive performance on DeepGlobe and SpaceNet datasets.

The authors also introduce a synthetic dataset that helps pretraining the autoregression model (need to clarify with the authors on that).
strength_and_weaknesses: ### Strengths
- competitive performance on two public datasets
- interesting synthetic dataset with path occlusion difficulty analysis

### Weaknesses
- more recent [relevant] related work that yields much improved results such as RNGDet (ignored) or VecRoad (different split, mentioned but ignored) is not compared against; what do you mean by "these baselines have been already shown to underperform though in the literature"? Who has shown that and where?
- small performance improvements on both datasets, seems to struggle at times, sometimes beaten by a 2017 paper
- convoluted pipeline, RGB segmentation >> point extraction from sampling >> auto-regression model >> muZero; feels like the whole system was glued in place to fix the problems from the previous step (e.g., the sampled keypoints are not perfect, throw in the auto-regression transformer model; its graphs are not that great, throw in the RL to figure out which graphs are usable); why not try predicting the graph directly?
- using RL for navigation / computing routes is not novel; for example, [1*] presented a similar concept, but with street views instead of satellite images.


___
[1*]Mirowski, P., Grimes, M., Malinowski, M., Hermann, K. M., Anderson, K., Teplyashin, D., ... & Hadsell, R. (2018). Learning to navigate in cities without a map. Advances in neural information processing systems, 31.
clarity,_quality,_novelty_and_reproducibility: Clarity/quality: 

As previously stated, the pipeline consist of a number of methods mashed together. Even an overview is missing - Fig 1 describes muZero modeling, which is a tiny bit of the whole pipeline. IMHO the authors need to test a modern architecture, such as RNGDet and check if RL improves on top of that. If not, well, RL is not the way to go for this task. Especially looking at the RNGNet results, I do not see compelling evidence the authors would be able to come close to those numbers, even with the complex system they built on top of the (already existing) graphs.

The paper has only minor text issues - e.g., Fig 6. "imporovement" x2, it is well written and structured. 

Novelty:

All in all, I do not think this paper in its current form is ICLR-worthy. It is more of an engineering work that struggles to marginally improve the results. Judging from the ablation study, they start from a terrible baseline, which again, in 2022, it is unacceptable. If my calculator is right, the 0.455 ALPS is the method without any improvements. Why? If the focus is the RL agent, why not start with the best graphs possible and improve over them? Is it something here that I am missing?

The synthetic dataset could be used to improve the results and show a compelling advantage for RL, but it's not. In fact, it's not even clear on how it's used, they claim the generation of a 'difficulty' score and training with LinkNet [a 2017 method], but only Figure 6 supports this claim, and no other attempts to find a meaningful use are described (e.g., is the autoregressive pretrained done on this dataset? What about other ideas, such as style transfer, or use to train the RL agent? Or is it this already done but not mentioned?).

Few recent approaches have attempted to upgrade the algorithms to modern pipelines (e.g., directly output vertices similar to [1*,2*], without intermediate keypoints/road segmentation, as in RNGDet/RNGDet++[3*- this paper is here only for future reference]), it would have been nice to adapt the framework, but no efforts were made.



Reproducibility:

The code is released in the supplementary material, it should be easy to reproduce the experiments.

___
[1*]Lazarow, J., Xu, W., & Tu, Z. (2022). Instance Segmentation With Mask-Supervised Polygonal Boundary Transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4382-4391).

[2*]Liang, J., Homayounfar, N., Ma, W. C., Xiong, Y., Hu, R., & Urtasun, R. (2020). Polytransform: Deep polygon transformer for instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 9131-9140). 

[3*]Xu, Z., Liu, Y., Sun, Y., Liu, M., & Wang, L. (2022). RNGDet++: Road Network Graph Detection by Transformer with Instance Segmentation and Multi-scale Features Enhancement. arXiv preprint arXiv:2209.10150.
summary_of_the_review: The paper provides an interesting alternative to all-in-one transformer-based architectures, but the latter are not compared and show significant improvements compared to (Batra et al., 2019), even though the split is different - RNGDet ( Xu et al., 2022)] or at least VectorNet (Tan et al., 2020). The ablation study is worrying to say the least - for an unknown reason, they start with a very poor baseline, despite the focus of the paper being the novel RL graph fixing method. I am waiting for the authors to clarify on this, I hope I haven't understood something right.

That being said, it proposes an interesting synthetic datasets approach for modelling occlusions/ambiguity. Nevertheless, it is underexploited - not clear why an old road detection was trained on it and how it helps the RL. It was only used in Figure 6 to check that LinkNet (a 5 year old method) is worse?
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper introduces a reinforcement learning framework for generating a graph as a variable-length edge sequence, where a structured-aware decoder selects new edges by simulating action sequences into the future. It reviews related works, explains the methodology, and presents the settings and results of experiments. The paper is followed by the Appendix with more details about the architecture, implementation, dataset creation and interoperability. There are also supplementary materials with the code which enable the reproducibility of experiments.
strength_and_weaknesses: Strengths:
- the introduced methodology seems to be quite novel
- the description of the methodology and experiments is comprehensive and the writing is quite good and clear
- supplementary materials with the code which enable the reproducibility of experiments 

Weaknesses:
- I have doubts regarding the importance of the use case (road network modelling from images) because nowadays, digital descriptions of road networks are usually generally available, but there are other works dealing with this topic and perhaps it's a good use case to demonstrate the introduced methodology. However, it would be good to demonstrate that the method can be useful in some other applications too.
clarity,_quality,_novelty_and_reproducibility: The quality of the presentation part is good and the writing is clear.
The quality from the methodological perspective seems to be good too.
The idea seems to be novel.
Supplementary materials with the code enable the reproducibility of experiments.

summary_of_the_review: The paper introduces a reinforcement learning framework for generating a graph as a variable-length edge sequence, where a structured-aware decoder selects new edges by simulating action sequences into the future. It reviews related works, explains the methodology, and presents the settings and results of experiments. The paper is followed by the Appendix with more details about the architecture, implementation, dataset creation and interoperability. The idea seems to be novel. The quality of the presentation part is good and the writing is clear. The quality from the methodological perspective seems to be good too. There are also supplementary materials with the code which enable the reproducibility of experiments. I have doubts regarding the importance of the use case (road network modelling from images) because nowadays, digital descriptions of road networks are usually generally available, but there are other works dealing with this topic and perhaps it's a good use case to demonstrate the introduced methodology. In the future, it would be good to demonstrate that the method can be useful in some other applications too. All in all, I like the paper and in my opinion, it's good enough to be published at ICLR.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper presents a road network prediction method using spatial graphs. The approach uses reinforcement learning. 

A number of graph measures are used to assess the method. 
strength_and_weaknesses: The paper is well-written and easy to follow. 
The literature is covered well. 
The method is very useful for road network extraction and presents a solution to a real-world problem. 
The results are strong with additional results provided in the appendix. 

The placement of figures is out of place and I don't see all figures referred to in the text. The text doesn't often tell the story alongside the figures. 

In remote areas, the target of this work, roads are often informal and exhibit a lot of variability e.g. width, surface, and use. Has this been considered?  A centerline wouldn't work in such cases. 

clarity,_quality,_novelty_and_reproducibility: The approach shows as novel and the results are good. I view high novelty. 

The code and data are referred to in the text. The work should be reproducible. 
summary_of_the_review: I thoroughly enjoyed this paper - the objective is clear, and the methodology and results well done.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper describes a solution for learning how to "draw" spatial networks of roads on top of satellite images. The solution is based on MCTS and MuZero. There are limited details in terms of the dataset used for training and how the system is actually deployed after training.

The system is evaluated with synthetic data and real-world satellite images. It is unclear if the proposed solution generalizes to images that are "different" from those used in the training set.

strength_and_weaknesses: Strengths

- The authors consider an interesting problem with clear practical applications.
- The method used to generate synthetic data is original and interesting.
- It appears that the solution proposed by the authors offers better performance compared to other state-of-the-art methods.

Weaknesses:

- As it is described in the paper, it seems that the proposed method needs the ground truth for actually receiving the reward. It is unclear if the methods is generalizable to different environments. This is not really evaluated in the paper (or at least it is not explicitly described in the paper).
- The reference to MuZero and the description of its application is quite hard to understand, especially in relation to the definition of the rewards. The same applies to the use of a MCTS, which is also only sketched in my opinion. 
- The reviewer was not able to understand how the RL is actually trained. In fact, it is unclear if the RL system is evaluated using the same dataset used for training. 
- The authors refer a learnable model of a dynamics network G, but it is difficult to map it to the actual inputs (and outputs) of the model.
- The values of the metrics used in the evaluation have very similar values. I believe that the authors should indicate the confidence intervals of these values. The confidence intervals might be overlapping.
- The authors should present the trade-off of the proposed methods in terms of computational complexity. Is the proposed method computationally expensive compared to the existing ones?
- There is limited information about the potential actual deployment post-training in terms of information needed as input. The authors should also clarify the outputs that are provided by the system in a clearer way in my opinion.
- The reviewer was not able to understand the concept of "dreamt tree search" presented in Figure 11.

clarity,_quality,_novelty_and_reproducibility: The idea of this paper is original and interesting, however, the authors do not clear show if and how it is generalizable to images with different characteristics. The reviewer was not able to understand the training process (and dataset) used by the authors. It is also unclear if there is any overlap between the training and evaluation datasets.

The presentation style of this work can be improved: in fact, some aspects are described in very generic terms and not in relation to the specific problem (in particular the use of MCTS and the application of MuZero in this situation).

In general, the reproducibility of the work is somehow limited since the description of the use of MuZero and MCTS is only sketched. The steps needed for the training and deployment of the system are also described in very abstract terms.
summary_of_the_review: Overall, this is an interesting paper, with potential practical applications. Unfortunately, some key information around the training and deployment of the system are missing. The descriptions of MCTS and MuZero are not completely clear (especially in relation the actions and inputs/outputs of the model).

The performance results do not show significant differences and it is difficult to evaluate them without confidence intervals. Other methods provide better performance in some situations.

It is very difficult to understand if the method generalizes to images that are different from the training set. The authors do not clearly specify if the training and test sets contain similar satellite images. 

This is a promising paper, but it is difficult to recommend acceptance in its current version.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper presents an approach for predicting the road networks directly as a graph structure from satellite images.  The approach predicts the graph structure sequentially and reinforcement learning is used to learn the model with MuZero style search.

This paper does not yet meet the bar of acceptance for mainly 2 reasons:
1. Presentation of the technical details of the method is not yet sufficient.  Even though the source code is provided, the authors should still have a clear technical presentation of their approach in the main paper with sufficient details for a knowledgeable reader to be able to reproduce the main idea of this work.  In its current form the reviewers agree that the clarity of the main paper is not yet sufficient to reach this requirement, as in particular the details of RL and the MuZero-style setup was not sufficiently described.
2. The presented results and baselines are a bit questionable.  In particular, the selected baselines for comparison and in particular the base setup used to build up this work seems to be a bit bad.  As reviewer d6d1 pointed out, a base setup only gets 0.455 ALPS which is far worse than any baselines.  This raises the question of why not build upon a better base model?  The proposed method seems to have sufficient potential and general enough to work on any base model, so the authors are strongly encouraged to consider this, which should also push the performance of this system even higher.
justification_for_why_not_higher_score: Clarity issue and the baseline issue described above.
justification_for_why_not_lower_score: Can't be lower.
Decision: Reject