Mastering Spatial Graph Prediction of Road Networks
Authors: Sotiris Anagnostidis
Aurelien Lucchi
Thomas Hofmann
Venue: Submitted to ICLR 2023 (2023)

Abstract: Accurately predicting road networks from satellite images requires a global understanding of the network topology. We propose to capture such high-level information by introducing a graph-based framework that simulates the addition of sequences of graph edges using a reinforcement learning (RL) approach. In particular, given a partially generated graph associated with a satellite image, an RL agent nominates modifications that maximize a cumulative reward. As opposed to standard supervised techniques that tend to be more restricted to commonly used surrogate losses, these rewards can be based on various complex, potentially non-continuous, metrics of interest. This yields more power and flexibility to encode problem-dependent knowledge. Empirical results on several benchmark datasets demonstrate enhanced performance and increased high-level reasoning about the graph topology when using a tree-based search. We further highlight the superiority of our approach under substantial occlusions by introducing a new synthetic benchmark dataset for this task.

Under review as a conference paper at ICLR 2023
MASTERING SPATIAL GRAPH PREDICTION OF ROAD NETWORKS
Anonymous authors
Paper under double-blind review
Prior policy
Current State
Simulate steps "app.........
?2
Pilor polay
Simulate next steps
X
Reward estimates 0: Value estimates
Figure 1: Our agent interacts with the currently generated spatial graph by proposing new edges to be added. A tree-based search produces a sequence of actions that maximizes a reward function based on complex geometries priors.
ABSTRACT
Accurately predicting road networks from satellite images requires global un- derstanding of the network topology. We propose to capture such high-level in- formation by introducing a graph-based framework that simulates the addition of sequences of graph edges using a reinforcement learning (RL) approach. In par- ticular, given partially generated graph associated with satellite image, an RI. agent nominates modifications that maximize cumulative reward. As opposed to standard supervised techniques that tend to be more restricted to commonly used surrogate losses, these rewards can be based on various complex, potentially non- continuous, metrics of interest. This yields more power and flexibility to encode problem-deye dependeen knowledge. Empirical results several benchmark datasets demonstrate enhanced performance and increased high-level reasoning about the graph topology when using tree-based search. We further highlight the superi- ority of our approach under substantial occlusions by introducing new synthetic benchmark dataset for this task.
INTRODUCTION
Road layout modelling from satellite images constitutes an important task of remote sensing, with numerous applications in and navigation. The vast amounts of data available from the commer- cialization of geospatial data, in addition to the need for accurately establishing the connectivity of roads in remote areas, have led an increased interest the precise representation of existing road networks. By nature, these applications require structured data types that provide efficient repre- sentations encode geometry, this case, graphs, de facto choice in domains such as computer graphics, virtual reality, gaming, and the film industry. These structured- graph representations are also commonly used to label recent road network datasets (Van Etten et al., 2018) and map reposi- tories (OpenStreetMapp contributors, 2017). Based these observations, for generating predictions directly as spatial graphs, allowing us to explicitly incorporate geometric constraints in the learning process, encouraging predictions that better capture higher-level dataset statistics.
propose new method
In contrast, existing methods for road layout detection, mostly rely pixel-based segmentation models that are trained on masks produced by rasterizing ground truth graphs. Performing pixel- wise segmentation, though, ignores structural features and geometric constraints inherent to the

---

Under review as a conference paper at ICLR 2023
problem. As result, minimum differences in the pixel-level output domain have significant consequences in the proposed graph, in terms of connectivity and path distances, as manifested by the often fragmented outputs obtained after running inference these models. In order address these significant drawbacks, we propose a new paradigm where we: (i) directly generate outputs as spatial graphs and (ii) formalize the problem by adding edges between key points. These key points can in principle come from any off-the-shelf detector that identifies road pieces with sufficient accuracy. Our generation process avoids having to resort to cumbersome post-pros post-processing steps (Batra et al., 2019; Montoya-Zegarra et al., 2015) or op- timize some surrogate objectives (Mättyus & Urtasun, 2018; Mosinska et al., 2018) whose relation to the desired qualities of the final prediction is disputed. Concurrently, the sequential decision-makinn strategy we propose enables us to focus interactively on different parts of the image, introducing the notion of current state and producing reward estimates for succession of actions. In essence, our method can be considered as a generalization of previous refinement techniques (Batra et al., 2019; Li et al., 2019b) with three major advantages: (i) removal of the requirement for greedy decoding, (ii) ability to attend globally to the current prediction and selectively target parts of the image, and (iii) capacity to train based demanding task task-specific metrics.
game where sequentially construct the output
More precisely, our contributions are the following:
We propose a novel generic strategy for training and inference in autoregressive models that re- moves the requirement of decoding according to a pre-defined order and refines initial sampling probabilities via a tree search.
We create a new synthetic benchmark dataset of pixel-level accurate labels overhead satellite images for the task of road network extraction. This gives us the ability simulate complex sce- narios with occluded regions, allowing us to demonstrate the improved robustness of our approach. We plan to release this dataset publicly.
We confirm the wide applicability of our approach by improving the performance of existing methods on the popular SpaceNet and DeepGlobe datasets.
RELATED WORK
013
0.1
880
620
Angle diegrees
Angle degrees
degree
junctions
Figure 2: Typical road network features (SpaceNet datasct). From left right: the distribution of angles between road segments leading the same intersection biased towards and degrees. parallel or perpen- dicular roads. The same also holds (b) for angles between random road pieces within ground distance of 400 meters. (c) Most road vertices belong to single road piece, with degree of 2. (d) The average number of intersections for areas of 400 meters by ground distance.
Initial attempts to extract road networks mainly revolved around handcrafted features and stochastic geometric models of roads (Barzohar & Cooper, 1996). Road layouts have specific characteristics, regarding radiometry and topology e.g. particular junction distribution, certain general orientation, and curvature (see Fig. 2), that enable their detection even in cases with significant occlusion and uncertainty (Hinz & Baumgartner, 2003). Modern approaches mostly formulate the road extrac- tion task as a segmentation prediction task (Lian al., 2020; Mattyus et al., 2015; Audebert al., 2017) by applying models such as Hourglass (Newell et al.. 2016) or LinkNet (Chaurasia & Culur- ciello, 2017). This interpretation has significant drawbacks when evaluated against structural losses, because of discontinuities in the predicted masks. Such shortcomings have been addressed by apply- ing some additional post-processing steps, such as high-order conditional random fields (Niemeyer et al., 2011; Wegner et al., 2013) or by training additional models that refine these initial predic- tions (Máttyus et al., 2017; Batra et al., 2019). Other common techniques include the optimization of ensemble of losses. Chu et al. (2019) rely directional loss and non- non-maximal suppres- sion as a thinning layer, while Batra et al. (2019) calculate orientations of road segments. Although such auxiliary losses somewhat improve the output consistency, the fundamental issue of producing
2

---

Under review as a conference paper at ICLR 2023
Segi Megrenalon
Transodac and eracigoo
Figure 3: (Left) Typical segmentation hased methods generate output graph by thresholding segmentation mask. which can however often lead to fragmented outputs. Predicting segmentation masks also makes impossible (right) to capture complex road interactions, such as overlapping roads at different elevations.
predictions in the pixel space persists. It remains impossible to overcome naturally occurring road network structures, e.g. crossings of roads in different elevations, see Fig. 3.
Previous failure cases have led to more intuitive conceptualizations of the task. Roadtracer (Bastani al., 2018), iteratively builds road network, similar depth-first search approach, while Chu et al. (2019) learn generative model for road layouts and then apply it as a prior top of seg- mentation prediction mask. Proposed graph-based approaches, encode the road network directly as graph, but either operate based on constrained step-size (Tan et al., 2020) to generate new vertices or operate on a single step (He et al., 2020; Bandara al., 2022), involving use-defined thresholding to post-process the final predictions. Most similar to our work, Li et al. (2019b) predict locations of key points and define a specific order traversing them, also similarly Xu al. (2022). Such autore- gressive models have been recently successfully applied with the use of transformers (Vaswani et al., 2017) in a range of applications (Nash et al., 2020; Para et al., 2021a;b; Xu et al., 2022) to model constraints between elements, while their supervised training explicitly requires tokens to be pro- cessed in a specific order. This specific order combined with the fact that only a surrogate training objective is used, introduces limitations, discussed further in the next section. In order eliminate this order requirement and to optimize based on the desired metric, while attending globally to the currently generated graph, we propose to use RL suitable alternative.
When generating discrete outputs, an unordered set of edges (Zaheer et al., 2017). it is challenging adapt existing learning frameworks train generative models (Para et al., 2021b). Instead of optimizing in the image space, however, we are interested in optimizing spatial structured losses by learning program heuristics, i.e. policies. RL has found success in the past in computer vision ap- plications (Le et al., 2021), but mainly an auxiliary unit with the goal of improving efficiency (Xu al., 2021) or as fine-tuning step (Qin et al., 2018). We instead rely on RL to produce the entire graph exploiting the ability of the framework for more high-level reasoning.
METHODOLOGY
We parametrize road network as graph G (V,E) with each vertex v; [x,, vilT V representing a key point on the road surface. The set of edges (v, v,) E E, corresponds to road segments connecting these key points. We can then generate probability distribution over roads by following a two-step process: i) generation of a set of vertices and ii) generation of a set of edges connecting them. Formally, for an image I, road network R is derived as:
R max P(V, & I) P(E [VZ))((VII)
V.E
The graph nodes typically correspond to local information in an image, and we therefore resort to a CNN-based model to extract key points, providing the set V" that sufficiently captures the information in the ground truth graph G. The construction of edges, however, requires higher-level reasoning that can cope with parallel roads, junctions, occlusions, poor image resolution, among other difficulties.
Considering probabilistic models over sequences and using the chain rule, we can factorize the joint distribution as the product a series of conditional distributions
Ne
P(E V.I:o)
II
P(ea(n) (<a(n)s V,Z),
(2)
71
3

---

Under review as a conference paper at ICLR 2023
Kay-points embedding
Edges ambedding (3)
Image processing (1)
fowerd
Feed lonax
Mas Intentee
Weodding
Figure 4: The base autoregressive model with its three main components. (1) A backbone image model (ResNet) that extracts features for each key point at different scales, along with a global image embedding. (2) A key point model embeds visual and location features of distinct key points. (3) An edge embedding model relates the current edge sequence with the respective key points. Each edge token (signalled with the tokens 'action-f") corresponds to index specifying the respective key point. A pair of such tokens designates an edge as its two endpoints, At the end of (3) we obtain new distribution over key points, that leads an incremental update the graph, after sampling.
where (<r(()) represents (I)) (2).)..... Cn(nn-1) and σ E SN₂ denotes the set of all permutations of the integers ,,2,...,NVe. Neg, with Ng the number of edges. For our work, we consider the setting where these sequences are upper bounded in length, i.e. Ng < Nmax. a reasonable assumption when dealing with satellite images of fixed size. Autoregressive models (ARMs) have been used to solve similar tasks in the past by defining a fixed order of decoding (Oord et al., 2016; van den Oord al., 2016; Nash al., 2020; Para et al., 202 In our case, this would correspond to sorting all key points by their x and locations and generating edges for each of them consecutively. We call this the autoregressive order. There are, however, two major drawbacks.
First, the evaluation metrics used for this task define a buffer region in which nodes in the ground truth and the predicted graph are considered to match. Therefore, a newly generated edge can be only partially correct, when only partially overlapping with the ground truth graph. This non- smooth feedback comes in clear contrast to the supervised training scheme of ARMs, minimization of the negative log-likelihood, that assumes perfect information regarding the key points' locations, i.e that the sets V and V' are the same. In practice, this condition is rarely met, as the exact be represented in arbitrarily many ways by subdividing long edges into smaller ones or due to small perturbation to key points' locations. It is thus imperative that our model can estimate the expected improvement of adding selected edges, which implicitly can also signal when to appropriately end the generation process.
spatial graph
Second, the requirement to decode according to the autoregressive order introduces bias and limits the expressiveness of the model (Uria et al., 2014). As a result, it can lead to failures in cases with blurry inputs or occlusions (Li et al., 2019b). Previous solutions include the use of beam search, either deterministic or stochastic (Meister et al., 2021). Beam search does not however eliminate the bias introduced in the selection order of the key points, while suffering from other deficiencies, such as degenerate repetitions (Holtzman et al., 2019; Fan et al., 2018). In order to address these shortcomings, we advocate for a permutation invariant strategy. We present a novel generic strategy, which improves autoregressive models without requiring significantly more computational cost.
3.1 AUTOREGRESSIVE MODEL
We start by introducing base autoregressive model, illustrated in Fig. Given an image and a set of key points, our model produces graph by sequentially predicting list of indices, corresponding to the graph's 'ss flattened, unweighted edge-list. Each forward pass produces probabilities over the set of key points, which leads to a new action after sampling. A successive pair of indices defines an edge as its two endpoints. A special sed-ueee token is reserved to designate the end of the generation process.
Following Wang et al. (2018); Smith et al. (2019), we begin by extracting visual features per key point, by interpolating intermediate layers of ResNet backbone the key points' locations, which
4

---

Under review as a conference paper at ICLR 2023
further augmented by position encodings of their locations. We then further process these fea- tures using two lightweight Transformer modules. The first transformer (Transformer in Fig. 4) encodes the features of the key points embeddings. The second transformer (Transformer II in Fig. 4) takes as input the currently generated edge list sequence, corresponding to the currently par- tially generated graph. Edges are directly mapped to the embeddings of their comprising key points, supplemented by position and type embeddings, to differentiate between them, as shown in Fig. 5 (a). An additional global image embedding, also extracted by the ResNet, is used to initialize the se- quence. The Transformer Il module produces a single hidden state, which is linked with the Ny + (corresponding to the provided key points, supplemented by the special end of the generation token) key points' embeddings by pointer network (Vinyals et al., 2015), via dot-product to generate the final distribution. This allows a variable number of actions that depends on the current environment state, instead of using fixed action space.
3.2 AUGMENTED SEARCH
In order to address the problems of greedy decoding (analysed Section 3), we frame our road extrac- tion task classical Markov- defines an environment, where the length the currently generated edge list determines the current step. Let Or, a and Tt correspond the observation, the action and the observed reward respec- tively, at time step t. The aim is to search for a policy that maximizes the expected cumulative reward over horizon T, i.e., max, Jmx) E. IEi= 7"r,] where 7 E (0, 1] indicates the discount factor and the expectation with respect to the randomness in the policy and the transition dynamics. We set the discount factor to due to the assumed bounded time horizon, and note that although the dynamics of the environment are deterministic, optimizing the reward remains challenging.
process (MDP). The generation of graph for every image
Each action leads to the selection of new key point, with new edges being added once every two actions. The addition of a new edge leads to a revision of the predicted graph and triggers an intermediate reward
Tt Speed, sc(9 Spred,..;
where se(9g,, Greed,) ) is a similarity score between the ground truth graph Ggr and the current estimate gpedd Discussion of the specific similarity scores used in practice is postponed for Section 3.3.
A proper spatial graph generation entails (i) correct topology and (ii) accurate location prediction of individual roads. For the latter, intermediate vertices of degree 2 are essential. We call a road segment (RS), an ordered collection of edges, between vertices of degree d(.) two (or a collection of edges forming a circle):
RS {Uis U1s2 Uis2)..... (UHH V1sa)) } s.t (Urs, U(s41) E E for i -1...,,-- k-1
d( ((U,)) = 2. for i = 2........ ...k 1, (d(v.)) + 2 and d(vr..) # 2 or U,s, = Vm.)).
During the progression of an episode (i.e. the sequential generation of graph), the topological nature of the similarity scores in Eq. 3 implics that the effect of each new edge to the reward will be reflected mostly once its whole corresponding road segment has been generated. To resolve the ambiguity in the credit assignment and allow our agent to look ahead into sequences of actions, rely on Monte Carlo Tree Search (MCTS) simulate entire sequences of actions. We use state-of-the-ar search-based agent, MuZero (Schrittwieser et al., 2020), that constructs a Icarnable model of the environment dynamics, simulating transitions in this latent representation and leading to significant computational benefits.
Specifically, MuZero requires three distinct parts (see also Fig. 5):
A representation function f that creates a latent vector of the current state h, = (o.). For this step, we use the autoregressive model, as shown in Fig. 4. Our current latent representation h, contains the graph's 's hidden state, along with the key points' embeddings used to map actions to latent vectors. As key points remain the same throughout the episode, image-based features (Components (1) and (2) in Fig. 4) are only computed once.
2. A dynamics network g. we use simple LSTM (Hochreiter & Schmidhuber, 1997), that predicts the effect of a new action by predicting the next hidden state and the expected reward: (he, f.) = go (h, ac). We can replace h with the latent representation 14-1, or its previous computed approximation h for tree search of larger depth larger than 1.
5

---

Under review as a conference paper at ICLR 2023
Aubeegnnssii Model
Oumer Nistien drasn
retwnds
Current edge
Nud Fa3der stago
Koy-ooot
Sq
Garor Irdean saal
Oroeding of par
retsadding
retunii
Edye langh
"PARI
The autorcgressive model generates a hidden state corresponding to the graph embedding. (a) When doing the edge decoder directly attends to the input key points' embeddings, augmented by position and type embeddings. (b) The prediction network pointer network allows an intuitive scale the action space by the number key points. (c) The dynamics network simulates trajectories by estimating new hidden states and rewards. For newly generated edges, it takes as input the embeddings of the new key points, but also the degree of the two vertices involved and the length of the newly proposed generated edge.
Figure 5:
the hidden state to produce value and policy predictions. A
3. A prediction network v,, that estimates the policy and the value for the current state (pr+ 1, Ut) (e()) We compute the policy via pointer network, described in Section 3.1. Value esti- mates are produced by a simple multi-layer network.
The dynamics network guides the search and evaluates the expected reward of actions. For every newly generated edge. also explicitly inform the network regarding the creation of new inter- sections and the expected relative change the overall road surface generated via embeddings (see Fig. 5). By using the dynamics network, we bypass the expensive call to the decoder module during the search, and can instead approximate small modifications in the latent representation directly. For our experiments, the dynamics network requires up to 90 times less floating-point operations to simulate trajectories, compared to using the edge embeddings' decoder. Effectively, our method does not involve significantly more computation budget compared to the base autoregressive model.
3.3 EVALUATION METRICS
We adopt the same evaluation metrics both a comparison between different methods but also the incremental rewards for our agent, by Eq. 3. We use the relaxed versions of precision, recall and in- tersection over union for pixel-level predictions mann et al., 1998; Wang et al., 2016). As graph-theoretic metrics we use APLS (Van Etten et al., 2018) and additionally include new metrics introduced in Citraro et al. (2020) that compare Paths, Junctions and Sub- Sub-graphs of the graphs in question, producing respectively precision, recall and fi scores. More details can be found in Appendix E
(CCQ) (Wiede-
EXPERIMENTS
Implementation details We resize images to 300 300 pixels, standardizing according to the training set statistics. For exploration, we initialize workers using Ray (Moritz et al., 2017) that execute episodes in the environment. For training, we unroll the dynamics function for la steps and use priority weights for the episode according to the differences between predicted and target values. Our algorithm can be considered as an approximate on-policy TD(A) (Sutton & Barto, 2018) due to the relatively small replay buffer. We reanalyse older games (Schrittwieser et al., 2020) to provide fresher target estimates. Unvisited graph nodes are selected based on an upper confidence score, balancing exploration with exploitation, similar to Silver al. (2018), We add exploration noise as Dirichlet noise and select actions based on a temperature-controlle sampling procedure, whose temperature is reduced during training.
Given the limited high-quality available ground truth labels (Singh et al., 2018) and to accelerate training, employ modifications introduced in EfficientZero (Ye et al., 2021). We investigate adding supervision to the environment model and better initialize Q-value predictions similar to the implementation of Elf OpenGo (Tian al., 2019). We further scale values and rewards using

---

Under review as a conference paper at ICLR 2023
Nydteri Sieee Froge
Sume
PEComee df IrerseD rg difccyyy
Figure 6: (Left) Examples of fragmented outputs from the LinkNet model under cases of significant occlusion and how our approach performs in these demanding circumstances. (Right) The performance gap between method and the same baseline is wide, based on topological spatial graph metries, for wide range of difficulties in the images. More details and examples are given in the supplementary material.
invertible transform inspired by Pohlen al. (2018). Here, predict support, as fully connected networks are biased towards learning low-frequency representations (Jacot et al., 2018) Selecting new actions involves generating simulations that can be done expeditiously given the small dimen- sion of the latent space and the modest size of the dynamics network. Finally, to generate key points, we skeletonize segmentation masks provided by any baseline segmentation model, by thresholding the respective segmentation masks produced and applying sip-siipliiiiation (Douglas & Peucker, 1973: Ramer, 1972). Selecting an appropriate threshold and subdividing larger edges guarantees that the generated set V" adequately captures most of the ground truth road network, leaving the complexity of the problem for our model to handle.
4.1 SYNTHETIC DATASET
We generate a dataset overhead satellite images of a synthetic town using CityEngine'. We randomly specify vegetation of varying height and width along the side walks of the generated streets, leading inadvertently to occlusions of varying difficulty. The simulated environment allows specifying pixel-perfect masks regarding both roads and trees occluding the road surface based on the provided camera parameters (Kong et al., 2020). We can hence tune the complexity of the task and quantify the benefits of approach for varying levels of difficulty. We defer regarding the generation process and dataset examples to the supplementary material.
details
We compare our method by training on our dataset LinkNet model (Chaurasia Culurciello, 2017), a popular segmentation model that has been widely used in the remote sensing community (Li al., 201 20)9a). Even in this synthetic and thus less diverse scenario, the deficiency of segmentation models to rely mostly on local information, with no explicit ability for longer-range interactions, is evident. Fig. 6, illustrates examples of such over-segmented predictions and how our approach can improve on them. We also define 'difficulty' attribute per synthetic satellite image, quantifying the occlusions as a percentage of the ground truth road mask covered. We observe considerable absolute improvement in topological metric scores when training our model on this synthetic dataset, compared to the LinkNet bascline, for varying image difficulty.
4.2 REAL DATASETS
We evaluate our method on the SpaceNet and DeepGlobe datasets. We use the same train-test splits in Batra et al. (2019) promote reproducibility, while results reported for the final combined graph on the original image scale. No pre-training on the synthetic dataset takes place. Further details regarding prepprocsssing available in the Appendix E.2
7

---

Under review as a conference paper at ICLR 2023
Table 1: Quantitative results for the SpaceNet and DeepGlobe datasets.
Methed
CCQ
11275
AFLS
Palb-Soodd
Jeackee
0000
4344.
21
pec.
100.
11
242- T 105.
f1
11
tlang Kuaee 2017)
01:4995
[111941
01,9W9
0.4143
(30822
IU5454
0 13090
015885
0.7155
0,2110
011458
0.21:7
Lenkhett (C (ctarres Cutezallo, 20123
[081000
0.7640
0.64879
0.8894
(80743
IL5943
0
0.6480
1U5599
0,808 0:780
0X3504
0.2576
Oneeration ara al, 2019
(1800)
0188822
0.3394
012884
018315
0.7175
(072880
0U/227
0.182
0:2991
80551
SPIN rea (lancars al, 2022)
0-157
02.622
0.9922
0.003
(L5422
0.7278
0:7285
06/220
(8.922
062827
0.7222
0.2837
Oue
0,7725
0.7852
0.99/0
1U5523
017223
0.7548
11.9431
187702
0.7924
0U7192
0.7922
Oneetatton (iuenu 2019
113857
036988
0.1641
03832
0 0.7034
05772
0.8292
Ours
1130200
0.7301
0.10%
10 (8:0500
03111
10:0554
038555
083335
** The prod proferdins Deepr
(f sigial Spacehin dasiout ieages
Alio
diffeiee from
reekedds owe Apprats
Ree best sovee Cieer worl hes sonee Eary inath oegonne in Siflwwr and wtt
4.2.1 COMPARISON TO BASELINES
We first verify that under the effectiveness of the proposed approach under an ideal scenario where the key points conditioned upon, correspond to the ones from the ground truth. In the interest of space we point the reader to Appendix A and Table 3. Subsequently, we move to the primary task of predicting spatial graphs without the ground-truth graph information but extract key points via the aforementioned process and train using the described topological metries directly. The previ- ous baselines are not applicable in this case, due to lack of ground truth information, so instead compare against the following; we explore powerful CNN architectures, by training a Segmentation model with ResNet backbone. We evaluate DeepRoadMapper (Máttyus et al., 2017). model that refines previous segmentation by adding connections along possible identified paths. As done by Batra al. (2019) we notice that in complex scenarios, the effect of this post-processing step is not always positive. We also evaluate against LinkNet (Chaurasia & Culurciello, 2017), and Orien- tation (Batra et al., 2019), which is trained to predict simultaneously road surfaces and orientation.
Table 2: Qualitative results of improved connectivity. We recommend zooming in for more details
Mappeer
Segmentation Kaassrr et al., 2017)
Orientation
(Mälyus et al., 2017)
(Batm et al., 2019)
Grand
Satllite Image
Quantitative results in Table I and visual inspection in Table 2, affirm that the global context and the gradual generation incite a better understanding of the scene, leading to consistently outperforming topological metric results compared to the baselines. We remark that predictions more topologically consistent with fewer shortcomings, such as double roads, fragmented roads, and over- connections. This is further supplemented by comparing the statistics of the predicted spatial graphs in Fig. 7. We further showcase the transferability of our model by employing it with no fine-tuning (apart from dataset-specific image normalization) on the DeepGlobe dataset. We can refine previous predictions by adding missing edges, leading to more accurate spatial graph predictions, as shown in Table This confirms our conjecture that road structures and geometric patterns are repeated across diverse cities" locations.
4.2.2 ABLATION STUDY
We experimented attending to image features for the two transformer modules by extracting per- patch visual features from the conditioning image Hime = img him .. as done in the Vision Transformer (Dosovitskiy et al., 2020). This did not lead to significant improvements, which we attribute to over-fitting In Fig. 8 we highlight the relative importance of some additional components for the final predictions. As efficiency is also of particular importance to us, we further visualize the effect of varying the simulation depth of the dynamics network during training. Surprisingly
8

---

500 Under review as a conference paper at ICLR 2023
Sround Tush Grevration Dus
Dorsty
130
236
300
820
26
40
60
60
Distance betweon Intersections (moters)
Num. Connected Comporents
Num. Intorsections
Node Degree
Figure 7: Comparison of generated graph statistics, following the same post- pesteprccssingg averaged across re- gions of 400 meters in ground distance. Orientation refers to the method of Batra et al. (2019).
032
Model
APLS
P-fl
J-f1
S-f1
Ours
0.6523
0.7431
0.7792
0.7922
039
autoregressive pretraining
-15.3%
-13.7%
-14,1%
-13.3%
visual festures key-tpinnts
-13.4%
-12.3%
tree-search during evaluation
-2.1%
15
-1,7%
-1.1%
+ cross attend image features
-0.2%
-0.4%
-0.7%
-0.3%
P: Path-bused, J: pnaction-buse.. S: Sub-graph-bawed
Figure 8: (Left) Ablation study on the SpaceNet dataset. front achieved by relating the number of simulations during training and the objective obj avg(APLS Path-based fl Junction-bused fl Sub-graph-based f1) for the same time budget.
(Right) The estimated Pareto
perhaps, our method performs consistently better than baselines, even for a small overall simulation length, as this already enables better policy approximations.
In Appendix A we provide incremental results for the task of predicting road networks based on optimal set of key points. In Appendix comparison to baselines based on the varying difficulty of the predicted underlying road networks. In Appendix we give more information regarding the generation of the synthetic dataset, while in Appendix D more information regarding the model architecture. Final in Appendix E we provide more implementation decisions, including details on exactly how key points and generated and how individual patch-level predictions are fused together. More examples of full environment trajectories are given in Appendix F. We stress that our method can act on partially initialized predictions, registering it also practical refinement approach top of any baseline. Initializing model according to the ARM model allows a moderately quick fine-tuning phrase. In combination with the learned environment model, which circumvents expensive calls the edge embedding model for each simulation step in the MCTS, allows us to train even on a single GPU.
provide insights concerning interpretability and further
5 CONCLUSIONS
We presented novel reinforcement learning framework for generating graph variable-length edge sequence, where a structured aware decoder selects new edges by simulating action sequences into the future. Importantly, this allows the model better capture the geometry of the targeted objects. This is confirmed by our experimental results, since our approach consistently produces more faithful graph statistics. One advantage of the proposed method that the reward function is based on (non-continuous) metrics that are directly connected to the application in question. Our approach does not require significantly more computational resources compared to state-of-the-ar supervised approaches, while in addition, it can be used to refine predictions from another given model. We also remark that the direct prediction of a graph enables the concurrent prediction of meta-information about the edges, including, for instance, the type of road (highway, primary or secondary street, biking lane, etc).
Our approach opens the door to several directions for future work. For example, we have assumed that a pre-defined model gives the location of key points, but one could instead augment the action space propose new key points' locations. Other promising directions include the direct prediction of deputdde graph primitives, e.g. T-junctions or roundabouts. Finally, we emphasize that approach is suitable to wide variety of applications where autoregressive models typically used, and it is of special interest when there is a need for complex interactions or constraints between different parts of object.

---

Under review as a conference paper at ICLR 2023
6 REPRODUCIBILITY STATEMENT
We have taken multiple steps to ensure reproducibility of the experiments. We refer the reader to Appendix E for complete description of the training protocol. We have also released the code as part of the supplementary material, including scripts on how to reproduce our results.
REFERENCES
Samira Abnar and Willem Zuidema. Quantifying attention flow in transformers. arXiv preprint vrXiv:2005.00923 2020.
David Acuna, Amlan Kar, and Sanja Fidler. Devil is in the edges: Learning semantic boundaries from noisy annotations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11075-11083, 2019.
Nicolas Audebert, Bertrand Le Saux, and Sébastien Lefèvre. Joint learning from earth observation and openstreetmap data get faster better semantic maps. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 67-75, 2017.
Wele Gedara Chaminda Bandara, Jeya Maria Jose Valanarasu, and Vishal M Patel. Spin road map- per: Extracting roads from aerial images via spatial and interaction space graph reasoning for autonomous driving. In 2022 International Conference Robotics and Automation (ICRA), pp. 343-350. IEEE, 2022.
Meir Barzohar and David Cooper. Automatic finding of main roads in aerial images by using geometric-stochastic models and estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(7):707-721, 1996.
Favyen Bastani, Songtao He, Sofiane Abbar, Mohammad Alizadeh, Hari Balakrishnan, Sanjay Chawla, Sam Madden, and David DeWitt. Roadtracer: Automatic extraction road networks from aerial images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4720-4728, 2018
Anil Batra, Suriya Singh, Guan Pang. Saikat Basu, CV Jawahar, and Manohar Paluri. Improved road connectivity by joint learning of orientation and segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10385-10393, 2019.
Cameron B Browne, Edward Powley, Daniel Whitehouse, Simon M Lucas, Peter Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. A survey of monte carlo tree search methods. IEEE Transactions Computational Intelligence and Al in games, 4(1):1-43, 2012.
Abhishek Chaurasia and Fugenio Culurciello. Linknet: Exploiting encoder representations for efficient semantic segmentation. In 20/7 IEEE Visual Communications and Image Processing (VCIP), pp. 1-4. IEEE, 2017.
Benjamin E Childs, James H Brodeur, and Levente Koesis. Transpositions and move groups in monte carlo tree search. In 2008 IEEE Symposium Computational Intelligence and Games, pp. 389-395. IEFF, 2008
Hang Chu. Daiqing Li, David Acuna, Amlan Kar, Maria Shugrina, Xinkai Wei, Ming- Yu Liu, An- tonio Torralba, and Sanja Fidler. Neural turtle graphics for modeling city road layouts. In Pro- ceedings of the IEFE/CVF International Conference Computer Vision, pp. 4522-4530 2019.
Leonardo Citraro, Mateusz Koziński, and Pascal Fua. Towards reliable evaluation of algorithms for road network reconstruction from aerial images. In European Conference on Computer Vision, pp. 703-719. Springer, 2020.
like Demir, Krzysztof Koperski, David Lindenbaum, Guan Pang, Jing Huang, Saikat Basu, For- est Hughes, Devis Tuia, and Ramesh Raskar. Deepglobe 2018: A challenge parse the earth through satellite images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 172-181, 2018.
10

---

Under review as a conference paper at ICLR 2023
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
David H Douglas and Thomas K Peucker. Algorithms for the reduction of the number of points required represent digitized line its caricature. Cartographica: the international journal for geographic information and geovisualization, 10(2):112-122, 1973.
Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. arXiv preprint arXiv: 04003, 2018.
Songtao He, Favyen Bastani, Satvat Jagwani, Mohammad Alizadeh, Hari Balakrishnan, Sanjay Chawla, Mohamed M Elshrif, Samuel Madden, and Mohammad Amin Sadeghi. Sat2graph: Road graph extraction through graph-tensor encoding. In European Conference on Computer Vision, pp. 51-67. Springer, 2020.
Stefan Hinz and Albert Baumgartner. Automatic extraction of urban road networks from multi-view aerial imagery. ISPRS journal of photogrammetry and remote sensing, 58(1-2):83-98, 2003.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text 2019.
degeneration. arXiv preprint
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and gen- eralization in neural networks. arXiv preprint arXiv:1806.07555 07572, 2018.
Pascal Kaiser, Jan Dirk Wegner, Aurélien Lucchi, Martin Jaggi. Thomas Hofmann, and Konrad Schindler. Learning aerial image segmentation from online maps. IEEE Transactions on Geo- science and Remote Sensing, 55(11):6054 6068, 2017.
Fanjie Kong, Bohao Huang. Kyle Bradbury, and Jordan Malof. The synthinel-1 dataset: a collection of high resolution synthetic overhead imagery for building segmentation. In Proceedings of the IEEF/CVF Winter Conference on Applications of Computer Vision, pp. 1814-1823, 2020.
Ngan Le, Vidhiwar Singh Rathour, Kashu Yamazaki, Khoa Luu, and Marios Savvides. Deep rein- forcement learning in computer vision: comprehensive survey. Artificial Intelligence Review, pp. 1-87, 2021.
Yuxia Li, Bo Peng, Lei He, Kunlong Fan, and Ling Tong. Road segmentation of unmanned aerial vehicle remote sensing images using adversarial network with multiscale context aggregation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7))2279- 2287, 2019a.
Zuoyue Li, Jan Dirk Wegner, and Aurélien Lucchi. Topological map extraction from overhead images. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 1715-1724, 2019b.
Renbao Lian, Weixing Wang, Nadir Mustafa, and Liqin Huang. Road extraction methods in high- resolution remote sensing images: A comprehensive review. JEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13:5489-5500 2020.
Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431-3440, 2015.
Gellért Máttyus and Raquel Urtasun. Matching adversarial networks. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 8024-8032, 2018.
Gellert Mattyus, Shenlong Wang, Sanja Fidler, and Raquel Urtasun. Enhancing road maps by pars- ing aerial images around the world. In Proceedings of the IEEE international conference on computer vision, pp. 1689-1697, 2015.
11

---

Under review as a conference paper at ICLR 2023
Gellért Máttyus, Wenjie Luo, and Raquel Urtasun. Deeproadmapper: Extracting road topology from acrial images. In Proceedings of the IEEE International Conference on Computer Vision, pp.
2017.
Clara Meister, Afra Amini, Tim Viera, and Ryan Cotterell. Conditional poisson stochastic beam search. arXiv preprint arXiv:2109, 1034, 2021.
Javier A Mlonoyy--egarral Jan D Wegner, L'u L'ubor Ladický, and Konrad Schindler. On the evaluation of higher-order cliques for road network extraction. In 2015 Joint Urban Remote Sensing Event (JURSE), pp. 1-4. IEEE, 2015.
Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, William Paul, Michael Jordan, and Ion Stoica. Ray: distributed framework for emerging ai applications. corr abs/1712.05889 (2017). arXiv preprint arXiv: 10w.1182....89,,, 2017.
Agata Mosinska, Pablo Marquez-Neila, Mateusz Koziński, and Pascal Fua. Beyond the pixel-wise loss for topology-awe delineation. In Proceedings of the IEEE conference on computer vision and patlern recognition, pp. 3136-3145, 2018.
Charlie Nash, Yaroslav Ganin, SM Ali Eslami, and Peter Battaglia. Polygen: An autoregressive generative model of 3d meshes. In International Conference on Machine Learning pp. 7220- 7229. PMLR, 2020.
Alejandro Newell, Kaiyu Yang. and Jia Deng. Stacked hourglass networks for human pose estima- tion. In European conference on computer vision, pp. 483-499. Springer, 2016.
Joachim Niemeyer, Jan Dirk Wegner, Clément Mallet, Franz Rottensteiner, and Uwe Soergel. Con- ditional random fields for urban scene classification with full waveform lidar data. In ISPRS Springer, 2011.
Conference on Photogrammetric Image Analysis, pp.
Aäron den Oord, Nal Kalchbrenner, Oriol Vinyals, Lasse Espeholt, Alex Graves, and Koray Kavukcuoglu. Conditional image generation with pixelenn decoders. In Proceedings of the 30th International Conference Neural Information Processing Systems, pp. 4797-4805, 2016.
OpenStreetMap contrribtoos.s Planet dump retrieved from https:/planet..om.orrg https:://///...... /www openst org, 2017.
Wamiq Para, Paul Guerrero, Tom Kelly, Leonidas Guibas, and Peter Wonka. Generative layout modeling using constraint graphs. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6690-6700, 2021a.
Wamiq Reyaz Para, Shariq Farooq Bhat, Paul Guerrero, Tom Kelly, Niloy Mitra, Leonidas Guibas, and Peter Wonka. Sketchgen: Generating constrained cad sketches. arXiv preprint arXiv:2/06.027// 2021b.
Tobias Pohlen, Bilal Piot, Todd Hester, Mohammad Gheshlaghi Azar, Dan Horgan, David Budden, Gabriel Barth-Maron, Hado Van Hasselt, John Quan, Mel Večerík, et al. Observe and look further: Achieving consistent performance on atari. arXiv preprint arXiv: 15933, 2018.
Pengda Qin, Weiran Xu, and William Yang Wang. Robust distant supervision relation extraction via deep reinforcement learning, arXiv preprint arXiv:/805.0992) 2018.
Urs Ramer. An iterative procedure for the polygonal approximation of plane curves. Computer graphics and image processing, 1(3):244-256, 1972.
Abdallah Saffidine, Tristan Cazenave, and Jean Méhat. Ued: Upper confidence bound for rooted directed acyelic graphs. Knowledge-Basee Systems, 34:26-33, 2012.
Julian Schrittwesser, loannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al. Mastering atari, go, chess and shogi by planning with leamed model. Nature, 588(7839):604-600) 2020.
12

---

Under review as a conference paper at ICLR 2023
David Silver, Thomas Hubert, Julian Schrittwieser, loannis Antonoglou, Matthew Lai, Arthur Guez, Mare Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play, Science, 362(6419):1140- 1144, 2018.
Suriya Singh, Anil Batra, Guan Pang, Lorenzo Torresani, Saikat Basu, Manohar Paluri, and CV Jawahar. Self-supervised feature learning for semantic segmentation of overhead imagery. In BMVC, volume 1, pp. 4, 2018.
Edward J Smith, Scott Fujimoto. Adriana Romero, and David Meger. Geometries: Exploiting geo- metric structure for graph-encoded objects. arXiv preprint arXiv: /901. /11111161,,111 2019.
Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.
Yong-Qian Ling-Qiang Tan, Shang-Hua Gao, Xuan-Yi Yi Li, Ming-Ming Cheng, and Bo Ren. Vecroad: Point- based iterative graph exploration for road graphs extraction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1910-8918, 2020.
Yuandong Tian, Jerry Ma, Qucheng Gong, Shubho Sengupta, Zhwoyuan Chen, James Pinkerton, and Larry Zitmick. Elf opengo: An analysis and open reimplementation of alphazero. In International Conference on Machine Learning, pp. 6244-6253, PMLR, 2019.
Benigno Uria, lain Murray, and Hugo Larochelle. A deep and tractable density estimator. In Inter- national Conference on Machine Learning, pp. 467-475. PMLR, 2014.
Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio. In 9th ISCA Speech Synthesis Workshop, pp. 125-125, 2016.
Adam Van Etten, Dave Lindenbaum, and Todd Bacastow. Spacenet: A remote sensing dataset and challenge series. arXiv preprint arXiv: 10XXv:8807.000 307.01 2018
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 1998-6008, 2017.
Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly, 15X0::1506.03144 2015.
Pointer networks.
arXiv preprint
Nanyang Wang. Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. Pixel2mesh: Generating 3d mesh models from single rgb images. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 52-67, 2018.
Shenlong Wang, Min Bai, Gellert Mattyus, Hang Chu, Wenjie Luo, Bin Yang, Justin Liang, Joel Cheverie, Sanja Fidler, and Raquel Urtasun. Torontocity: Seeing the world with a million eyes. arXiv preprint 1rX.v:1612.00444
2016.
Jan D Wegner, Javier A Montoya-Zegarra, and Konrad Schindler. A higher-order erf model for road network extraction. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 1698-1705, 2013.
Christian Wiedemann, Christian Heipke, Helmut Mayer, and Olivier Jamet. Empirical evaluation of automatically extracted road axes. Empirical evaluation techniques in computer vision, 12: 172-187, 1998.
Jingtao Xu, Yali Li, and Shengjin Wang. Adazoom: Adaptive zoom network for multi-scale object detection large scenes. arXiv preprint arXiv:2106. 10409, 2021.
Zhenhua Xu, Yuxuan Liu, Lu Gan, Yuxiang Sun, Xinyu Wu, Ming Liu, and Lujia Wang. Rngdet: Road network graph detection by transformer in aerial images. IEEE Trunsactions on Geoscience and Remote Sensing, 60:1-12, 2022.
Weirui Ye, Shaohuai Liu, Thanard Kurutach, Pieter Abbeel, and Yang Gao. Mastering atari games with limited data. arXiv preprint 11Xiv:2111.00211 2021.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan Salakhutdinov, and Alexander Smola. Deep sets. arXiv preprint arXiv07703.061 14, 2017.
13

---

Under review as a conference paper at ICLR 2023
Table APLS metric and perplexity (bits of information per edge) results for the SpaceNet dataset with provided key points that adequately capture the ground truth graph. Autoregressive order is defined in Section 3, while random order entails random permutation between edges and the order of key points within edges. To calculate perplexity for our method we use the initial predicted policy distribution, without any additional search.
Metric
Method
Random
Cls
GCN
ARM
Ours
APLS
0.008
0,310
0,488
0.743
0.778
Bits per edge: Autoregressive order
8.743
0.733
4.636
Bits per edge: Random order
8.743
30.84
4.565
A MORE EXPERIMENTS
We first assess the performance of our proposed method in an ideal scenario where the key points, correspond to the ones from the ground truth. To hinder training and inference, we insert additional key points as (1) random intermediate points between known edges and (2) randomly sampled loca- tions in the images. Here, our assumption in Section 3 that the set V suffices to generate the ground truth graph, holds by construction. We compare our method against several baselines that learn to connect edges between key points, using the same feature extraction pipeline, described Sec- tion 3.1,
model. Cls is a classification network that predicts for all pairs of key points a value {0, 1} corresponding to the existence of an edge. GCN implements a graph neural network that pre- diets directly the adjacency matrix. We also present autoregressive version of our model ARM, that is trained with cross-entropy loss to predict the pre-defined ordered sequence of key points. We use this model to initialize ours. Results are presented in Table 3.
As expected, the ARM model achieves a low perplexity score when evaluated against the corre- sponding sequence, ordered according the autoregressive order, but suffers in predicting the edges when in random order. The ARM underperforms because of frequent early terminations and the im- plicit inability to revisit key points, what the desired final metric is concerned, here APLS. Even though our model is developed upon this autoregressive model, it generates tokens in an arbitrary arrangement. Reward and value estimates enable different training scheme that deeply correlates with the desired objective.
B INTERPRETABILITTY
We visualize attention (of the Transformer II module), using the attention flow proposed in Abnar & Zuidema (2020), in Fig. 9, To create attention scores per edge, we aggregate scores for the pair of tokens that define each edge. New predictions lay increased attention already generated junctions, parallel road segments, and other edges belonging to the same road segment.
Figure 9: We visualize attention by aggregating attention scores for key points that form the same edge. White denotes the latest edge added, for which the attention scores are calculated. Colours indicate the amount of attention any current edge in the graph.
We also compare APLS results achieved by varying the difficulty of the ground truth images in terms of the total number of junctions (vertices with a degree greater than 2) and in terms the average length of road segments that are present, in Fig. 10. Our method explicitly captures information
14

---

Under review as a conference paper at ICLR 2023
4.5
408
Figure 10: APLS improvement of our method, compared to the Orientation method Batra et al. (2019), based on images capturing a ground distance area of x00 400 meters of the SpaceNet dataset. (Left) We vary the number of junctions present, and (right) the average road segment length (in meters).
garding the degree of the key points during the search, while it can encode better global information, even across larger distances. It is not surprise perhaps then, that it outperforms the baselines more convincingly as the difficulty of the ground truth road network increases.
Finally, we visualize example of an imagined rollout trajectory at single step of our algorithm in Fig. 11. During a single inference step, our method uses tree search to look ahead into sequences of actions in the future. For example, have chosen relatively smaller number of simulations (10) for better visual inspection. We also show the corresponding environment states reached, which are, however, not explicitly available to the model, as it is searching and planning using learned model of the environment.
C DATASET CREATION
We use CityEngine, 3D modelling software for creating immersive urban environments. We gener- ate simple road network and apply a rural city texture on the created city blocks, provided by Kong et al. (2020). We then uniformly generate trees of varying height and size along the side walks of the generated streets. We then iteratively scan the generated city by passing a camera of specific orientation and height. We repeat the same process after suitable modifications to the texture, for the generation of the street masks, as well as the vegetation masks, that correspond to only the plants along the side walks. Some examples of the generated images are provided in Fig. 12. We note that additional occlusion can be caused by the relation of the camera with the 3D meshes correspond- ing buildings. These occlusions are, however, not captured by our generated masks, and we can expect them to contribute partially to the fragmented segmentation results.
We train a segmentation- based model. LinkNet, as our baseline. We rasterize the ground truth graph to create pixel-level labels and train by maximizing the intersection over union, which is commonly done in practice. We note that there choice of the line-width with which the ground truth graph is rasterized. A large width achieves better results terms connectivity of the predicted graph but results in poorer accuracy in the final key points' locations. Furthermore, when providing large width, areas in the image with more uncertainty, e.g. vegetation that is not above a road segment, are also predicted as road networks with high certainty, leading spurious, disconnected road segments. To highlight the advantages of our method compared to this baseline and in order to promote more meaningful predictions, we select relatively smaller width.
tradeoff between the nature of the predictions and the
15

---

Under review as a conference paper at ICLR 2023
Figure 11: Example of a "dreamt" tree search of model (we recommend zooming in for more details). Here, images correspond to the environment observations by following the respective se- quence of actions, which are, however, not given as input to the model but only portrayed for visual purposes. The model has only access to the original observation and "dreamt" trajectories on the learned latent space. The root of the search respond to the children that attain the highest estimated value. We also provide reward and value estimates of our model based on the current latent representation. We perform a smaller number of simulations into the future, for visual purposes.
is indicated by the colour red. Orange nodes cor-
D ARCHITECTURE DETAILS
As image backbone model, we use 18 for the synthetic dataset and a ResNet-50 for the real dataset experiments. We extract features four different scales, after each the ResNet layers. To extract features for each key point, we interpolate the backbone feature maps based on the key points' locations. We use different learned embeddings based on the actual key points' locations. For the key points embedding model, a dropout rate 0.15. We use layer normalization and GELU activation functions.
transformer encoder with 16 self-attention layers and
For the edge-embeddings model, use the respective key points embedding, along with learned position and type embeddings, which we all sum together. As aforementioned, we can initialize the current edge sequence based previous predictions, allowing our model refine any initial prediction provided. Again, we use the same transformer architecture with 16 self- atf-attention layers, and dropout rate of 0.15.
16

---

Under review as a conference paper at ICLR 2023
Figure 12: Samples from the synthetic dataset. We generate images and the corresponding street mask, overlaid with the colour red, along with the masks of plants that are occluding the ground truth road network, overlaid with the colour green.
Finally, the architecture of the dynamics network and the value prediction network are shown in Fig. 13. For the value estimation, also provide the current environment step, execute steps in an environment with a bounded time horizon.
Hidden state
Current step
Key point embedding
Previous hidden
Vertex type embedding
Edge lengtt embeddiing
Linear (512 x 258)
Linear (256 x 256)
GELU
LSTM Input 1024
Linear (256 x 256)
Linear (256 x 256)
Hidden size: 512
GELU
GELU
Ouiput size: 256
Linear (258 x 256)
Linear (256 x 256)
GELU
Linear (256
Linear (256 601)
Linear (256 601)
Linear (256 256)
Next hidden state
Reward estimate
Value estimate
Figure 13: Architecture details of the dynamics network and the value prediction network. Reward and value are determined by predicting support of size 601. Final scalar values are calculated by an invertible transform of this support, similar to Pohlen et al. (2018).
17

---

Under review as a conference paper at ICLR 2023
Vegas
Paris
Shanghai
Khartoum
Figure 14: Example images sampled from the four cities of the SpaceNet dataset. Images from dif- ferent cities exhibit different regularity in their road networks. The quality of the overhead satellite images may also vary.
E IMPLEMENTATION DETAILS
E.1 EVALUATION METRICS
APIS (Van Etten et al., 2018) constitutes a graph theoretic metric that faithfully describes routing properties. APLS is defined
1 N2
Σ min {1. Peres - Poj
APLS =
(4)
Poroz
P0192 C00
where and s' denote source node and its closest point on the predicted graph if such exists within buffer. Np denotes the number of paths sampled and Pervs the length of the shortest path between two nodes. Similarly, the Too Long Too Short (TLTTS) metric (Wegner al., 2013) compares lengths of the shortest paths between randomly chosen points of the two graphs, classifying them as infeasible, correct, or too-long or too-short (2/+2s) if the length the path on the predicted graph does not differ by more than threshold (5%) compared to the ground truth path. Since small perturbations to the predicted graph can have larger implications to pixel-level predictions, the definitions of precision, recall and intersection over union relaxed in Wiedemann et al. (1998); Wang et al. (2016) leading to the metrics
(CCQ). Still, some types of errors, such double roads or over-connections, not penalized from the above metrics (Citraro et al., 2020). We therefore additionally include new metrics introduced Cit- raro al. (2020) that compare Paths, Junctions and Sub-graphs of the graphs in question, producing respectively precision, recall and fi scores. For the final similarity score used in Eq. 3, we use linear combination of the aforementioned metrics, more details are available in the supplementary material.
E.2 DATASET INFORMATION
We use the following datasets to train our models, i.c. baselines and our newly proposed RL agent.
SpaceNet (Van Etten et al., 2018) includes road network of over 8000 Km over four different cities: Vegas, Paris, Shanghai, and Khartoum, where the complexity and quality, and regularity of the road network depend on the city of origin. Satellite images of 1300 x 1300, corresponding to a ground resolution of 30cm per pixel. We split the 2780 total images into crops of size 400 x 400 with overlap of 100 pixels for training. To better highlight the diversity of the satellite images from these four different locations, we have included some randomly sampled examples in Fig. 14.
provided
pixel resolution
DeepGlobe (Demir et al., 2018) contains satellite images from 3 different locations with pixel- level annotations. Images have resolution of 1024 1024, with ground resolution of 50cm per pixel. We crop the 6226 images into tiles, leading to a similar ground truth resolution per pixel compared to SpaceNet.
18

---

Under review as a conference paper at ICLR 2023
Example of how key points are generated. We start by evaluating a segmentation model (left) and extracting the predicted graph (middle). We then over-sample vertices along edges to enlarge the action space (right).
Figure 15:
E.3 TRAINING DETAILS
At each MCTS search step, we perform several simulations from the root state s for number of steps k L.... and select action that maximizes the upper confidence bound (Silver et al., 2018),
a² arg max ((s.a) + P(s,a) N (s, (c1 log Es N(. b) + 02 + ))
N(s.u)
02
where ((s a), Q(s, a), P(s,a) corresponds to the visit counts, mean values and policies, calcu- lated by the current search statistics. Constants C1 C2 balance exploration and exploitation. Based on state * and a selected action a², dynamics network. We update the mean values based on bootstrapped values of the estimated value functions and rewards. We experimented with training the reward and value support predictions with both mean squared error (MSE) and cross cooss-y loss. We opted for MSE because of its stability. For more depth description of the training scheme of MuZero we recommend Schrittwieser et al. (2020) and Ye et al. (2021).
state s and reward job are estimated through the
As hinted in the main text, we train using intermediate rewards, linear combination of topo- logical metrics. We experimented using a variety of different scores and metrics, but ended up using APLS, Path-based fl, Junction-based fl and Sub- Sub-graph-based fl at a relative scale of (0.35, 0.25, 0.25, 0.15). We found the Sub-graph-based fl to be more sensitive to small pertur- bations and therefore weighted it less in the final combination. The metrics mentioned above are highly correlated, examined in Batra et al. (2019). This correlation, though, holds when compar- ing the final predictions. Intermediate incremental rewards are more independent, so we still found it useful to mixture of them. Initially, to let our network learn basic stable rewards, segmentation prediction mask as target. That means that we train our model to predict the graph that can be extracted after post-processing the segmentation model's prediction.
the
After pre-training the autoregressive model, we experimented with fine-tuning using RL with two different learning rates, where slower by factor (0 1] rate was chosen for the pre-trained modules. Here, we noticed that the model still performed better than the ARM baseline. As it has trouble though to escape the autoregressive order, compared to the single learning rate model. results are less optimal.
We finally note that by avoiding type and position encoding in the Transformer II module, we can ensure the embedded graph is permutation invariant regarding the sequence of edges and the order of key points within an edge. Our search graph can then be formulated into a directed acyelic graph, circumventing unnecessary division of the search space (Browne et al., 2012; Childs et al., 2008), enabling more efficient sampling (Saffidine al., 2012). These updated search statistics are cumbersome to compute, though, and we found significant efficiency improvement. They do, however, confirm our model's potential ability to handle the input graph as an unordered set, as the problem suggests.
E.4 PRODUCING KEY POINTS
We initially train a segmentation model for predicting pixel-level accurate masks of the road net- work. For this step, use any model from the literature. We extract the predicted graph by
19

---

Under review as a conference paper at ICLR 2023
skeletonizing the predicted mask and simplifying the graph by smoothing threshold. We then sample intermediate vertices along the largest in terms of ground length edges, to enlarge the action space. We illustrate toy example of such process in Fig. 15. To accelerate inference, we can also initialize our prediction graph based on the provided segmentation mask. In such a case, our method closer resembles previous refinement approaches. We additionally remove edges of con- nected components with small overall size and edges belonging to roads segments leading to dead ends (that means vertices of degree one), keeping though the corresponding key points in the en- vironment state. Thus, if our model deems the existence of the respective edges necessary, it can add them once more. We plan further investigate augmenting the action space with the ability to remove edges in future work, that would not require such pre-processing strategy.
E.5 COMBINING PREDICTIONS
When creating the final per image prediction, we initially simply generated predictions on non- overlapping patches and fused them together. To overcome small pixel location differences in the predicted graphs, we fuse by rasterizing the individual graphs in the pixel domain with a line width larger than What we found more successful was to perform inference on overlapping patches and to initialize the currently predicted graph based on the predictions made so far. This is particularly useful, as road segments are often close to the boundaries of our cropped image. Individual inference and simple fuse can often lead to over-connected predictions. We visualize toy example of such process in Fig. 16.
Inference
Simple Fusion
Rasterize
Extract Graph
Figure 16:
Toy example of how inference is performed in larger images. We start by cropping the image to overlapping patches. (Top) Naive fusion leads to over-connections because of perturbation in the key points' locations. (Bottom) We initialize subsequent graph predictions based on the key points and edge predictions of the so far generated output from previous patches if such exists.
For the segmentation baselines, unless specified in their respective documentation, we perform infer- ence by cropping images to overlapping patches and normalizing the final predicted mask based on the number of overlapping predictions per pixel location. We also pad images around their boundary, as done in Acuna et al. (2019). We note some small differences in the final scores for the Orientation model (Batra et al., 2019) and the SpaceNet dataset, compared to the ones in Citraro et al. (2020). We assume these are an outcome of different chosen parameters for the calculation of metrics. We keep these parameters fixed when calculating scores for all methods.
20

---

Under review as a conference paper at ICLR 2023
E.6 MORE COMPARISONS WITH BASELINES
We elaborate more on the evaluation method on Sat2Graph. The authors provided predictions cor- responding only center crop of the original SpaceNet dataset images. For each 400 400 pixel image, predictions are made for the center 352 x 352 area of the image. One could expect slightly better results if trained in the same conditions but that the gap does still seem large enough show the merits of our approach.
Other baselines like Neural turtle graphics (Chu al., 2019) and Topological Map Extraction (Li et al., 2019b) do not have an implementation available. We do not compare against VecRoad (Tan al., 2020) or RoadTracer (Bastani al., 2018), as different datasets were used for the current evaluations. These baselines have been already shown to underperform though in the literature, by methods that we are comparing against.
MORE EXAMPLES
We showcase in Fig. and Fig. more examples of the environment state progression, for the synthetic dataset.
21

---

Under review as a conference paper at ICLR 2023
Surtin
Figure 17: Example of environment progression for the synthetic dataset. Key points' locations are shown in blue. By over-sampling initial segmentation predictions as shown in Fig. 15, we can generate key points in possibly occluded areas of the image.
22

---

Under review as a conference paper at ICLR 2023
Serting sire
Lewe
Los wwwer serehard
Figure 18: Example of environment progression for the synthetic dataset. Generating the same edge twice (between key points and 7), although unintuitive, does not lead to different final predicted graph.
23


Reviews:
[{"review_id": "uDRolg563V", "reviewer": "Reviewer_ro23", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper introduces a reinforcement learning framework for generating a graph as a variable-length edge sequence, where a structured-aware decoder selects new edges by simulating action sequences into the future. It reviews related works, explains the methodology, and presents the settings and results of experiments. The paper is followed by the Appendix with more details about the architecture, implementation, dataset creation and interoperability. There are also supplementary materials with the code which enable the reproducibility of experiments.", "summary_of_the_review": "The paper introduces a reinforcement learning framework for generating a graph as a variable-length edge sequence, where a structured-aware decoder selects new edges by simulating action sequences into the future. It reviews related works, explains the methodology, and presents the settings and results of experiments. The paper is followed by the Appendix with more details about the architecture, implementation, dataset creation and interoperability. The idea seems to be novel. The quality of the presentation part is good and the writing is clear. The quality from the methodological perspective seems to be good too. There are also supplementary materials with the code which enable the reproducibility of experiments. I have doubts regarding the importance of the use case (road network modelling from images) because nowadays, digital descriptions of road networks are usually generally available, but there are other works dealing with this topic and perhaps it's a good use case to demonstrate the introduced methodology. In the future, it would be good to demonstrate that the method can be useful in some other applications too. All in all, I like the paper and in my opinion, it's good enough to be published at ICLR.\n", "strength_and_weaknesses": "Strengths:\n- the introduced methodology seems to be quite novel\n- the description of the methodology and experiments is comprehensive and the writing is quite good and clear\n- supplementary materials with the code which enable the reproducibility of experiments \n\nWeaknesses:\n- I have doubts regarding the importance of the use case (road network modelling from images) because nowadays, digital descriptions of road networks are usually generally available, but there are other works dealing with this topic and perhaps it's a good use case to demonstrate the introduced methodology. However, it would be good to demonstrate that the method can be useful in some other applications too.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "hzTqBMTuyIZ", "reviewer": "Reviewer_d6d1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a reinforcement learning approach to road vectorization. In contrast with previous works, the task is modeled as generating a graph as a variable-length edge sequence. \n\nThe pipeline consists of multiple stages: semantic segmentation from RGB images (this results in binary maps), a transformer-based autoregression model (results in a collection of road graphs) and the RL part- muZero adapted for road graph extraction. It achieves competitive performance on DeepGlobe and SpaceNet datasets.\n\nThe authors also introduce a synthetic dataset that helps pretraining the autoregression model (need to clarify with the authors on that).", "summary_of_the_review": "The paper provides an interesting alternative to all-in-one transformer-based architectures, but the latter are not compared and show significant improvements compared to (Batra et al., 2019), even though the split is different - RNGDet ( Xu et al., 2022)] or at least VectorNet (Tan et al., 2020). The ablation study is worrying to say the least - for an unknown reason, they start with a very poor baseline, despite the focus of the paper being the novel RL graph fixing method. I am waiting for the authors to clarify on this, I hope I haven't understood something right.\n\nThat being said, it proposes an interesting synthetic datasets approach for modelling occlusions/ambiguity. Nevertheless, it is underexploited - not clear why an old road detection was trained on it and how it helps the RL. It was only used in Figure 6 to check that LinkNet (a 5 year old method) is worse? \n", "strength_and_weaknesses": "### Strengths\n- competitive performance on two public datasets\n- interesting synthetic dataset with path occlusion difficulty analysis\n\n### Weaknesses\n- more recent [relevant] related work that yields much improved results such as RNGDet (ignored) or VecRoad (different split, mentioned but ignored) is not compared against; what do you mean by \"these baselines have been already shown to underperform though in the literature\"? Who has shown that and where?\n- small performance improvements on both datasets, seems to struggle at times, sometimes beaten by a 2017 paper\n- convoluted pipeline, RGB segmentation >> point extraction from sampling >> auto-regression model >> muZero; feels like the whole system was glued in place to fix the problems from the previous step (e.g., the sampled keypoints are not perfect, throw in the auto-regression transformer model; its graphs are not that great, throw in the RL to figure out which graphs are usable); why not try predicting the graph directly?\n- using RL for navigation / computing routes is not novel; for example, [1*] presented a similar concept, but with street views instead of satellite images.\n\n\n___\n[1*]Mirowski, P., Grimes, M., Malinowski, M., Hermann, K. M., Anderson, K., Teplyashin, D., ... & Hadsell, R. (2018). Learning to navigate in cities without a map. Advances in neural information processing systems, 31.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "gQtswHKvdte", "reviewer": "Reviewer_BpnF", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a road network prediction method using spatial graphs. The approach uses reinforcement learning. \n\nA number of graph measures are used to assess the method. ", "summary_of_the_review": "I thoroughly enjoyed this paper - the objective is clear, and the methodology and results well done. ", "strength_and_weaknesses": "The paper is well-written and easy to follow. \nThe literature is covered well. \nThe method is very useful for road network extraction and presents a solution to a real-world problem. \nThe results are strong with additional results provided in the appendix. \n\nThe placement of figures is out of place and I don't see all figures referred to in the text. The text doesn't often tell the story alongside the figures. \n\nIn remote areas, the target of this work, roads are often informal and exhibit a lot of variability e.g. width, surface, and use. Has this been considered?  A centerline wouldn't work in such cases. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "cN12H8cDQZ", "reviewer": "Reviewer_zSe9", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper describes a solution for learning how to \"draw\" spatial networks of roads on top of satellite images. The solution is based on MCTS and MuZero. There are limited details in terms of the dataset used for training and how the system is actually deployed after training.\n\nThe system is evaluated with synthetic data and real-world satellite images. It is unclear if the proposed solution generalizes to images that are \"different\" from those used in the training set.\n", "summary_of_the_review": "Overall, this is an interesting paper, with potential practical applications. Unfortunately, some key information around the training and deployment of the system are missing. The descriptions of MCTS and MuZero are not completely clear (especially in relation the actions and inputs/outputs of the model).\n\nThe performance results do not show significant differences and it is difficult to evaluate them without confidence intervals. Other methods provide better performance in some situations.\n\nIt is very difficult to understand if the method generalizes to images that are different from the training set. The authors do not clearly specify if the training and test sets contain similar satellite images. \n\nThis is a promising paper, but it is difficult to recommend acceptance in its current version.", "strength_and_weaknesses": "Strengths\n\n- The authors consider an interesting problem with clear practical applications.\n- The method used to generate synthetic data is original and interesting.\n- It appears that the solution proposed by the authors offers better performance compared to other state-of-the-art methods.\n\nWeaknesses:\n\n- As it is described in the paper, it seems that the proposed method needs the ground truth for actually receiving the reward. It is unclear if the methods is generalizable to different environments. This is not really evaluated in the paper (or at least it is not explicitly described in the paper).\n- The reference to MuZero and the description of its application is quite hard to understand, especially in relation to the definition of the rewards. The same applies to the use of a MCTS, which is also only sketched in my opinion. \n- The reviewer was not able to understand how the RL is actually trained. In fact, it is unclear if the RL system is evaluated using the same dataset used for training. \n- The authors refer a learnable model of a dynamics network G, but it is difficult to map it to the actual inputs (and outputs) of the model.\n- The values of the metrics used in the evaluation have very similar values. I believe that the authors should indicate the confidence intervals of these values. The confidence intervals might be overlapping.\n- The authors should present the trade-off of the proposed methods in terms of computational complexity. Is the proposed method computationally expensive compared to the existing ones?\n- There is limited information about the potential actual deployment post-training in terms of information needed as input. The authors should also clarify the outputs that are provided by the system in a clearer way in my opinion.\n- The reviewer was not able to understand the concept of \"dreamt tree search\" presented in Figure 11.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject