Reinforcement learning for instance segmentation with high-level priors
Authors: Paul Hilt
Edgar Kaziakhmedov
Maedeh Zarvandi
Sourabh Bhide
Maria Leptin
Constantin Pape
Anna Kreshuk
Venue: Submitted to ICLR 2023 (2023)
Keywords: Instance Segmentation
Reinforcement Learning
Biomedical Imaging

Abstract: Instance segmentation is a fundamental computer vision problem which remains challenging despite impressive recent advances due to deep learning-based methods. Given sufficient training data, fully supervised methods can yield excellent performance, but annotation of groundtruth data remains a major bottleneck, especially for biomedical applications where it has to be performed by domain experts. The amount of labels required can be drastically reduced by using rules derived from prior knowledge to guide the segmentation. However, these rules are in general not differentiable and thus cannot be used with existing methods. Here, we revoke this requirement by using stateless actor critic reinforcement learning, which enables non-differentiable rewards. We formulate the instance segmentation problem as graph partitioning and the actor critic predicts the edge weights driven by the rewards, which are based on the conformity of segmented instances to high-level priors on object shape, position or size. The experiments on toy and real data demonstrate that a good set of priors is sufficient to reach excellent performance without any direct object-level supervision.

Under review as a conference paper at ICLR 2023
REINFORCEMENT LEARNING FOR INSTANCE SEGMEN- TATION WITH HIGH-LEVEL PRIORS
Anonymous authors Paper under double-blind review
ABSTRACT
Instance segmentation is fundamental computer vision problem which remains challenging despite impressive recent advances due to deep learning- based meth- ods. Given sufficient training data, fully supervised methods can yield excellent performance, but annotation of groundtruth remains a major bottleneck, especially for biomedical applications where it has to be performed by domain experts. The amount of labels required can be drastically reduced by using rules derived from prior knowledge to guide the segmentation. However, these rules are in general not differentiable and thus cannot be used with existing methods. Here, we revoke this requirement by using stateless actor eritic reinforcement learning, which enables non-differentiable rewards. We formulate the instance segmentation problem as graph partitioning and the actor critic predicts the edge weights driven by the rewards, which are based on the conformity of segmented instances to high-level priors on object shape, position or size. The experiments on toy and real data demonstrate that a good set of priors is sufficient to reach excellent performance without any direct object-level supervision.
INTRODUCTION
Instance segmentation is the task of segmenting all objects in an image and assigning each of them different id. It is the necessary first step to analyze individual objects in scene and is thus of paramount importance in many computer vision applications. Over the recent years, fully supervised instance segmentation methods have made tremendous progress both in natural image applications and in scientific imaging, achieving excellent segmentations for very difficult tasks (Chen, Wang, and Lee et al. 2017).
Qiao 2021
large corpus of training images hard to avoid when the segmentation method needs to take into account the full variability of the natural world. However, in many practical segmentation tasks the appearance of the objects can be expected to conform to certain rules that are known priori. Examples include surveillance, industrial quality control and especially medical and biological imaging applications where full exploitation of such prior knowledge is particularly important as the training data is sparse and difficult acquire: pixelwise annotation of the necessary instance-level groundtruth for a microscopy experiment can take weeks or even months of expert time. The use of shape priors has a strong history this domain Delgado-GGonzale et al. 2014 Osher and Paragios 2007], but the most powerful learned shape models still require groundtruth (Oktay et al. 2018)) and generic shapes are hard to combine with the CNN losses and other, non-shape, priors. For many high-level priors it has already been demonstrated that integration of the prior directly into the CNN loss can lead superior segmentations while significantly reducing the necessary amounts of training data (Kervadec et al. 2019). However, the requirement of formulating the prior as a differentiable function poses a severe limitation on the kinds of high-level knowledge that can be exploited with such approach. Our contribution addresses this limitation and establishes a framework in which rich set of non-differentiable rules and expectations can be used to steer the network training.
To circumvent the requirement of a differentiable loss function, we turn to the reinforcement learning paradigm, where the rewards can be computed from a non-differentiable cost function. We base our framework stateless actor-critic setup (Pfau and Vinyals Vinyals(0016), providing one of the first practical applications of this important thcoretical construct. In more detail, we solve the instance segmentation problem as agglomeration of image superpixels, with the agent predicting the weights of the edges in

---

Under review as a conference paper at ICLR 2023
the superpixel region adjacency graph. Based on the predicted weights, the segmentation is obtained through (non- differentiable) graph partitioning. The segmented objects are evaluated by the critic, which learns to approximate the rewards based on object- and image-level reasoning (see Fig. D
The main contributions of this work can be summarized as follows: (i) we formulate instance segmen- tation as a RL problem based on stateless actor-critic setup, encapsulating the non-differentiable step of instance extraction into the environment and thus achieving end-to-end learning; (ii) we do not use annotated images for supervision and instead exploit prior knowledge on instance appearance and morphology by tying the rewards to the conformity of the predicted objects to pre-defined rules and learning to upproximate the dinfeeeerret)) reward function with the critic; (iii) we introduce a strategy for spatial decomposition of rewards based on fixed-sized subgraphs to enable localized supervision from combinations of object- and image-level rules. (iv) we demonstrate the feasibility of our approach on synthetic and real images and show an application to two important segmentation tasks in biology. In all experiments, our framework delivers excellent segmentations with no supervision other than high-level rules.
2 RELATED WORK
Reinforcement learning has so far not found significant adoption in the segmentation domain. The closest to our work are two methods in which RL has been introduced to learn sequence of segmentation decision steps as a Markov Decision Process. In the actor critic framework of Araslanov, Rothkopf, and Roth 2019 the actor recurrently predicts one instance mask at time based on the gradient provided by the critic. The training needs fully segmented images as supervision and the overall system, including complex. In Jain et al. 2011 the individual decision steps correspond to merges of clusters while their sequence defines hierarchical agglomeration process on superpixel graph. The reward function is based on Rand index and thus not differentiable, but the overall framework requires full (super)pixelwise supervision for training.
LSTM sub-network between the encoder and the decoder, is fairly
Reward decomposition was introduced for multi agent RL by Sunchag et al. 2017 where a global reward is decomposed into a per agent reward. Bagnell and Ng 2006 proves that a stateless RL setup with decomposed rewards requires far less training samples than a RL setup with a global reward. In Xu et al. 2219 reward decomposition is applied both temporally and spatially for zero-shot inference on unseen environments by training on locally selected samples to learn the underlying physics of the environment.
The restriction to differentiable losses is present in all application domains of deep learning. Common ways to address are based on soft relaxation of the loss that can be differentiated. The relaxation can be designed specifically for the loss, for example, Area-under-Curye (Eban et al. 2017) for classification or Jaceard Index (Berman, Triki, and Blaschko 2018) for semantic segmentation. These our
approaches are not directly applicable image-level priors, which should be combined without handcrafting an approximate loss for each case. More generally, but still for concrete task loss, Direct Loss Minimization has been proposed in Y. Song et al. 2016] For semi-supervised learning of a classification or ranking task, Discriminative Adversarial Networks have been proposed as a means to learn an approximation to the loss (Santos, Wadhawan, and Zhou 2017). Most generally, Grabocka, Scholz, and Sehmidt-Thiemn 2019 propose to train a surrogate neural network which will serve as a smooth approximation of the true loss. In our setup, the critic can informally be viewed as a surrogate network as it learns to approximate the priors through the rewards by Q-learning.
aim to use variety of object- and
Incorporation of rules and priors is particularly important in biomedical imaging applications, where such knowledge can be exploited to augment or even substitute scarce groundtruth annotations. For example, the shape prior is explicitly encoded in popular nuclear (Schmidt et al. 2018) and cellular (Stringer et al. (021) segmentation algorithms based on spatial embedding learning Learned non-linear representations of the shape are used in Oktay et al. 2018, while in Hu et al. 2019 the loss for object boundary prediction is made topology-ave
awwrr. aware. Domain-specinf priors can also be exploited in prtccessesinng by graph partitioning (Pape et al. 2019). Interestingly, the energy minimization procedure underlying the graph partitioning can also be incorporated into the learning step (Abbas and Swoboda 2021; Maitin-Shepard et al. 2016) J. Song al. 2019).
2

---

Under review as a conference paper at ICLR 2023
b) Agent
U-Net
GNN
Assumilation
Action
L(-)
State
Reward
d) Environment
(
Figure I: Interaction of the agent with the environment: (a) shows the state, which is composed the image and superpixels; (b) depicts the agent, which consists of the actor and critic networks well as the feature extractor that computes the node input features; (c) given the state, the agent performs the actions by predicting edge weights the graph; (d) the environment, which includes the image, superpixels, graph and graph partitioning based on the weights predicted through agent actions: (e) rewards obtained by evaluating the segmentation arising from the graph partitioning, based pre-defined and data dependent rules. The rewards are given back to the agent where they are used for training.
3 METHODS
The task of instance segmentation can be formalized as transforming an image x into a labeling that maps each pixel to label value. An instance corresponds the maximal of pixels with the same label value. Typically, the instance segmentation problem is solved via supervised learning, ie. using training set with groundtruth labels ÿ. Note that y invariant under the permutation of label values, which makes it difficult to formulate instance segmentation in fully differentiable manner. Most approaches first predict a "soft" representation with a CNN, e.g. affinities (Gao et al. [2199] Lee et al. 2017] Wolf et al. (020), boundaries (Beier et al. 2017 Funke et al. 2018)) or embeddings (De Brabandere, Neven, and Van Gool 2017] Neven al. 2019) and apply non- differentiable post-prooe post-processing, such agglomeration (Bailoni et al. 2019 Funke al. 20018) density clustering (Comaniciu and Meer 2002] McInnes and Healy 2017) or partitioning (Andres et al. 2012), obtain the instance segmentation. Alternativlyy, proposal-based methods predict bounding-bow per instance and then predict the instance mask for each bounding- box (He et al. 2017). Furthermore,
the common evaluation metrics for instance segmentation (Meilä 2003 differentiable.
Rand 1971 are also not
Our main motivation explore RL for the instance segmentation task is to circumvent the restriction to differentiable losses and regardless of the loss to make the whole pipeline end-to-end even in presence of non-differreniialle steps that transform pixelwise CNN predictions into instances.
We formulate the instance segmentation problem using a region adjacency graph G = (V, E). where the nodes V correspond superpixels (clusters of pixels) and the edges É connect nodes that belong to spatially adjacent superpixels. Given edge weights W, the instance segmentation is obtained by partitioning the graph, here using an approximate multicut solver (Kernighan and Lin 1970). Together, the image data, superpixels, graph and the graph partitioning make up the environment of our RI. setup. Based on the state of ε, the agent A predicts actions a. Here, the actions are interpreted edge weights W and used to partition the graph. The reward is then computed based on this partitioning. Our agent A a statcless actor-critic (Haarnoja et al. 2018), represented by two graph neural networks (GNN) (Gilmer al. 2017). The actor predicts the actions a based on the graph and its node features F. The node(superpixel) features are computed by pooling together the corresponding pixel features based the raw image data.
We compute the node features with a UNet (Ronneberger, Fischer, and Brox 2015) that takes the image input and outputs feature vector per pixel. These features spatially averaged over the
3

---

Under review as a conference paper at ICLR 2023
superpixels to obtain F. The feature extractor UNet is part of the agent A, thus training it end-to-end with the actor and critic networks (Fig. [ and fixed feature extractor to combine the learned features with hand-crafter ones.
In low data regimes it is also possible to use a pre-trained
Crucially, the reinforcement setup enables us to use both a non-differentiable instance segmentation step and reward function, by encapsulation of the "pixels instances" step in the environment and learning policy based on the rewards with the stateless actor critic.
3.1 STATELESS REINFORCEMEN LEARNING SETUP
Unlike most RL settings (Sutton and Barto 2018). our approach does not require an explicitly time dependent state: the actions returned by the agent correspond to the real-values edge weights in [0, 1]. which are used compute the graph partitioning. Any state can be reached by single step from the initial state and there exists no time dependency in the state transition. Unlike Jain et al. [011] we predict all edge values at once which allows us to avoid the iterative strategy Araslanov, Rothkopf, and Roth 2019 and deliver and evaluate complete segmentation in every step. Hence, we implement a stateless actor critic formulation.
Stateless RL was introduced in Pfau and Vinyals 2016 to study the connection between generative adversarial networks and actor critics, our method is one of the first practical applications this concept. Here, the agent consists of an actor, which predicts the actions and critic, which predicts the action value Q (expected future discounted reward) given the actions. The stateless approach simplifies the action value: estimates the reward for single step instead of the expected sum of discounted future rewards for many steps. We have explored a multi-step setup as well, but found that yields inferior results for our application; details can be found in the App. A.8| Furthermore. we compute sub- -o-g-raphh rewards instead of relying on single global reward in order to provide a more localized reward signal (see Section 3.2 for details).
The actor corresponds to single GNN, which predicts the mean and variance of a Normal distribution for each edge. The actions a are determined by sampling from this distribution and applying a sigmoid to the result to obtain continuous edge weights in the value range [0, 1]. The GNN takes the state (G, F) as input arguments and its graph convolution for the 7th node defined as in Gilmer et al.
2017
f: = Y
(
Σ JEN())
o (fr
INOI
INOI
where Yn as well as dh are MLPs, (. is the concatenation of vectors and N(i) is the set of neighbors of node i. The gradient of the loss for the actor is given by:
V Tahettor V
a
Σ (og(="((l=)) - Qsg
SG
#gEG
AEsg
This loss gradient is derived following Haarnoja et al. 2018 We adapt it to the sub- graph reward structure by calculating the joint action probability of the policy x" over each sub-graph sg in the set of all sub-graphs SG. Using this loss to optimize the policy parameters 0 minimizes the Kullback-Lan Leibler divergence between the Gibbs distribution of action values for each sub-graph Qug (s. a) and the policy with respect the parameters 0 of the policy. is trainable temperature parameter which is optimized following the method introduced by Haarnoja et al. 2018
The critic predicts the action value Qay for each sub-graph sg E SG. consists of a GNN Q (s, a) that takes the state (G, F) as well the actions predicted by the actor as input and predicts feature vector for each edge. The graph convolution from Equation [ slightly modified:
= 1Q
J.
sQ (Ji: JEN())
Σ
again 7Q and Φo are MLPs. Based on these edge features Qag is predicted for each sub- -ab-graph via an ML.. Here, we use set of subgraph sizes (typically, 6, 12, 32, 128) to generate supervison signal
4

---

Under review as a conference paper at ICLR 2023
01
O2
reward value
0.6
1.00
num
0.80
CHT value
80 0.70
The graph is subdivided into sub- graphs, each sub-graph is highlighted by a difffernt color. All sub-graphs have the same number of edges (here 3). Overall, we use a variety of sizes covering different notions of locality.
Figure 2:
Figure 3: An example reward landscape Cir- cle Hough Transform (CHT) rewards. High rewards are given the overall number of predicted objects is not too high and if the respective object has large CHT value.
for different neighborhood scales. A given MLP is only valid for a fixed graph size, we employ different MLP for each size. The loss for the critic is given by:
Ceritic =
Σ #gEG
(Q. (s, r)²
|SG
Minimizing this loss with respect to the action value function's parameters minimizes the difference between the expected reward and action values (s, a).
3.2 LOCALIZED SUB-GRAPH REWARDS
In most RL applications a global scalar reward is provided state transition. In our application of graph-based instance segmentation, it instead desirable to introduce several more localized rewards in order to learn from reward for the specific action, rather than global scalar. Here. reward decomposition is natural because we evaluate the segmentation quality per object and can use the object scores to provide localized reward. In order formalize this idea, have designed our actor critic (Section 3.1 to learn from sub-graph rewards.
A good set of sub- sub-graphs should fulfill the following requirements: each sub- -ob-graph should be connected so that the input to the MLP that computes the activation value for the sub-graphs is correlated. The size of the sub-graphs should be adjustable and all sub-graphs should be extracted with the exact same size to be valid inputs for the MLP. The union of all sub-graphs should cover the complete graph so that each edge contributes should overlap to provide smooth sum of action values. We have designed Alg. to extract a set of sub-graphe -obggraphs according to these requirements. Fig. shows an example sub-graph decomposition.
least one action value Q Qag. The sub-graphs
While some of the rewards used in our experiments can be directly defined for sub-graphs, most are instead defined per object (see App. A.2 for details on reward design). We use the following general procedure to map object-level rewards to sub-graphs: first assign to each superpixel the reward of its corresponding object. The reward per edge determined by the maximum value its two incident superpixels' rewards. The edge rewards averaged to obtain the reward per sub-graph.
By taking the maximum we assign the higher score to edges whose incident superpixels belong to different objects, because they probably correspond to correct split. Note that the uncertainty in the assignment of low rewards can lead to noisy reward signal, but the averaging of the edge rewards the sub-gn mub-graphs and the overlaps between the sub- -o--graphs smooth the rewards. We have also explored different actor critic setup that can use object level rewards directly, with no sub sub-graph extraction and mapping. However, this approach yields inferior results, App. A.3 for details.
5

---

Under review as a conference paper at ICLR 2023
Superphoot
Fostures
Prediction
Reward:
Prediction
MWS baseline
(a) Reinforcement learning output.
(b) Mutex watershed baseline.
Figure 4: Synthetic data. a) top left to right: groundtruth segmentation, raw data, superpixels and visualization of the actions (merge actions in green, split actions in red). Bottom left right: pre- trained pixel embeddings, superpixel edges, segmentation result and visualization of the rewards (light green for high rewards, dark red for low rewards, b) comparison of segmentation from method and the mutex watershed.
EXPERIMENTS
We evaluate our approach on three instance segmentation problems: one synthetic and two real. For a proof-of-prin -rrnn---riiniple. we construct synthetic dataset with cireular shapes on structured background, showing how our framework can exploit simple geometric priors. Next, we apply the method to a popular microscopy benchmark dataset for nucleus segmentation (Caicedo et 2019). Finally, we consider challenging biological segmentation problem with boundary-labeled cells. Here, we evaluate both learning restricted to prior rules and mixed supervision combining rule-based and direct rewards computed from groundtruth annotations. The problem setup, network architectures and hyperparameters are reported in detail App. A.9
4.1 SYNTHETIC DATA: CIRCLES ON STRUCTURED GROUND
We create synthetic images of circles on structured background and segment this data using only simple geometric rules. Superpixels were generated with the mutex watershed (Wolf et al. 2020) applied to the gaussian gradient of the image. Here, demonstrate that the actor critic can be trained without any direct object-level supervision and apply a simplified setup with a fixed pixel feature extractor, pre-trained through self-supervision (see App. A.I).
The object-level reward is based on the Circle Hough Transform (CHT) (Hassanein et al. 2015). is combined with estimate for the total number of objects in the image The global reward gives useful gradients during early training stages: when too few potential objects found in the prediction, low reward can be given the tentative background object. If too many potential objects are found, a low reward can be given to all the foreground objects with a low CHT value. The surface created by the per-object and global reward shown in Fig. The exact reward computation can be found in A.11
additional global reward.
Fig. 4 shows the output of all algorithm components on a sample image. We also computed results with the mutex watershed (Wolf et al. (2200), a typical algorithm for boundary based instance segmentation in microscopy. Texture within objects and structured background are inherently difficult for
algorithms, but our approach exploit higher-level reasoning along with low-level information and achieve a good segmentation.
4.2 REAL DATA: NUCLEUS SEGMENTATION
Nuclei are very frequent target of instance segmentation in microscopy. which is also reflected in the large amount publicly available annotated data. The availability of training data sparked the development of popular pre-trained solutions, such generalist UNet (Falk al. 2019) or StarDist

---

Under review as a conference paper at ICLR 2023
(Schmidt et al. 2018) and CellPose (Stringer et al. 202
which both have an (implicit) shape prior. Also, due to ubiquity of nuclei in microscopy, detailed prior knowledge exists on their shape and their appearance under different stainings. The experiments in this section aim to answer the following questions: i) given fully annotated groundtruth images for training, is there an advantage in using our RI. formulation with object-level rewards compared to commonly used fully supervised baselines? ii) given superpixels that can be combined into the correct solution, but no other direct supervision, can our approach learn to combine the superpixels correctly only from high-level rules? iii) what happens if superpixels are suboptimal? For data, we turn to the dataset of Caicedo et al. 2019 and select images that contain nuclei of medium size (175 for training and 22 for test).
Features are learned end-to-end. In the unsuper- vised setting, we compute the reward by combining several object descriptors: eccentricity, extent, mi- nor diameter, perimeter, solidity as well as mean, maximum and minimum intensity per object. The object reward is then given by the normalized sum of square distances of these quantities and their expected value. Objects larger than 15,000 pixels considered to belong the background and are not assigned a reward. Since the superpixels serve as fixed input into our model that do not get modi- fied, the accuracy of our segmentations is bound by their accuracy. To investigate their influence, we evaluate our approach with three different sets of superpixels: "GT", where we intersect the superpix- els with the groundtruth object masks to ensure that a correct segmentation can be recovered, "UNET", where compute the superpixels using predic- tions of a pre- pre-trained U-Net as an edge detector and "RAW", where we only take into account the raw image data. See 1e12 for more details on superpix- els and object descriptors.
Method
Superpixel
mAP
IoU50
1oU/5
1NNet
0.710
0.900
0.756
StarDSst
0.645
0.938
0,736
Cellpase
0.666
0.951
0.776
1Ner MC
0.674
0.806
0,702
ours (sup.)
GT
0.766
0.907
0,799
Ctsu
0.554
0,763
0.579
ours (unsup.)
GT
0,743
0.916
0,787
(unsup-) curs (unsup.)
UNET
0.671
0.872
0,704
RAW
0.785
Ap gt sp gt
UNET
RAW
0.554
0.969
0.505
Table 1: Nuclei segmentation: for mAP and IoU higher values are better. Methods above the first middle line were trained fully super- vised. Methods below the first middle line were trained without groundtruth, the results below the second middle line indicate the quality of the superpixels projected to the groundtruth (best possible result that can be achieved with the given superpixels).
Tab. summmrizzs the results, with a comparison to popular generalist pre-trained nuclear segmenta- tion methods: StarDist (Schmidt et al. 20188), Cellpose (Stringer et al. (221) and UNet (Falk et al. 2019). For StarDist and Cellpose, we use the pre-trained models provided with the papers. The UNet trained on the same images StarDist, the instance segmentation is recovered either by applying connected components to the boundary-subtrace comparison conditioned on a particular set of superpixels, by using the UNet boundary predictions and superpixels described above as input to Multicut graph-based agglomeration ("UNet+ + MC"). Otsu threshold serves as a simple unsupervised baseline (Otsu) 1979), where binarizing the image is followed by connected components to obtain the instance segmentation.
foreground prediction ("UNet") or, to obtain a
For the first question, we train our pipeline fully supervised ("ours (sup.)") as described in App. A. 10 we use pixelwise groundtruth, but can also exploit our RI. formulation where the loss is assigned to individual objects through the -on-differentiale graph agglomeration step. Here, our method performs better than all baselines without RL, so there is clearly an advantage using object-level supervision (as also demonstrated recently for non-RL setups, e.g. by Wolny, Yu, et al. 2021).
For the other two questions, we train our method using only rule-based rewards ("ours (unsup.)" ""). Given superpixels from which the groundtruth image can be recovered ("GT"), we then achieve better segmentation quality than the fully supervised baselines and the gap in performance between our unsupervised and supervised approach is smaller than the gap to the runner-up baseline. Of note, our unsupervised model also outperforms the "UNet + MC" baseline using the same "GT" superpixels, so its performance cannot be explained just by the use of groundtruth in superpixel generation. Example results and failure cases are shown in App. A.12
In the third experiment, we use a pretrained UNet as an edge detector to create superpixels of "medium" quality and again obtain strong results, outperforming StarDist, CellPose and UN with "GT" superpixels. Finally, with our worst set of superpixels obtained directly from the raw data, the method can learn to exploit the rules, but is clearly hindered by the superpixel quality.
7

---

Under review as a conference paper at ICLR 2023
4.3 REAL DATA: CELL SEGMENTATION
Biomedical applications often require segmentation of objects of known morphology arranged in regular pattern (Thompson T992). Such data presents the best use case for our algorithm as the reward function can leverage such priors. We address a cell segmentation problem from developmental biology, where cells often follow stereotypical shapes and arrangement: 317 drosophila embryo images from Bhide et al. 2020 including 10 with expert annotations used test data. Note that several tre-trained networks are available for cell segmentation in light microscopy (Chamier et al. 2021;; Stringer et al. [021] Wolny, Cerrone, et al. (2220); however, they produce sub-par results on this data due to experimental differences.
The rewards express that the cells are arranged in a radial pattern, with the average size known from other experiments (see Fig. [5) We set high reward for merging superpixels that certainly belong to the background (close the image boundary or
Mehhol
VI
VI menge
VI VT
1,266
0.672
0,594
ours
2.213
0.839
1,374
(semissup))
1.634
0,753
0.901
center). For background edges near the foreground area, we modulate the reward by the circularity of the overall foreground contour. For the likely foreground edges, we compute object-level rewards by fitting rotated bounding box to each object and comparing its radii and orientation to template values. We use weight profile based on the known embryo width to combine object and background rewards (App. [.
ours (handerasted)
2.523
0.987
UNe + MC
3,361
3.019
0.342
contrastive
4,440
1.155
3.28
Table 2: Cell segmentation results, measured by variation of information (VI) (Meilà 2003). This entropy-basee metric is commonly used to evaluate crowded segmentations in microscopy. We also report its merge and split component that measure the pear:/under-segmentalie error respectively. Lower values are better.
More formally, the rewards are calculated as fol- lows: for each edge, define the position h as the average of the centers of the two incident superpixels. Given the image center c, the radius of a circle that approximately covers the foreground j and the (maximal) image border position m, we use gaussian kernel (C)- for weighting and define edge reward redge:
Here 7, 7) and 8 are normalization constants. The kernel function in Eq. [ determines the back- ground probability of an edge;
K.
[|--|| 7
(1 a),
if S
a constitutes (5) reward that favors merges. It is scaled by the background probability. The object rewards ro are found by fitting a rotated bounding box to (6) the object and then comparing orientation and extent expected values known from previous experiments. They are mapped to edge rewards Tol, using the maximum value of the two incident objects.
Tbg
K
[mm
(1 a); otw
11
|1/
j|l
rfg K.
max(rai,
8
Redge =ggg + Fbg
We pre-compute superpixels by using boundary predictions as input to watershed seeded from local maxima. We use the UNet from Wolny, Cerrone, et al. 2020), which was trained on roughly similar images. As it was trained on plant cells in different microscope modality, its prediction is far from perfect, especially around the inner circle, see Fig. Eddee prediction"". We combine the learned node features with hand-crafted features: the normalized polar coordinate of the superpixel center and the normalized superpixel size. Fig. 5 shows visualisations of the learned and hand-crafted features. Interestingly, the learned features converge to a representation that resembles a semantic segmentation of boundaries.
Tab. shows the results: "ours" is the method described above; for "ours (semisup.)" we train a model that additionally receives direct supervision from groundtruth for single patch using the reward from App. A. and for "ours (handcrafted)) we only use the hand-crafted features and not the learned features. We include the UNet from Wolny, Cerrone, et al. 2020 with Multicut for instance segmentation ("UNet + MC") as well as the method of De Brabandere, Neven, and Van Gool 2017 trained on the same data Wolny, Cerrone, et al. 2020 ("contrastive") only 10 images of the dataset are annotated, we cannot efficiently finetune any of the popular cell segmentation networks on this dataset. We also project the superpixels their respective groundtruth
baselines. Since
8

---

Under review as a conference paper at ICLR 2023
Ground Truth
Raw
Edge Prediction
Superpixels
Actions
PCA Features a)
b)
c)
Prediction
Rewards
Figure Cell segmentation experiment. Top left to right: groundtruth segmentation; raw data; boundary predictions; superpixel sver-segmentttion; visualization for the actions on every edge (green = merge action, red split action). Bottom left to right: a) handcrafted features per superpixel; b) learned features averaged over superpixels; c) learned features per pixel; Multicut segmentations; visualization of the rewards (light green high reward, dark red low reward). For all features, we use the first 3 PCA components for visualisation. White arrows point to remaining errors.
cluster ("sp g(") to indicate the best possible solution that can be achieved with the given superpixels. Our approach clearly outperforms the baseline methods trained on the data from Wolny, Cerrone, et al. 2020 While predictions are not perfect (white arrows in Fig. prior rules turn out to be sufficient to assemble most cells correctly. The remaining errors are caused by objects not fully conforming the priors ("bent" rather than straight oval cells) by very weak boundary prediction. Furthermore, we see that the learned features significantly improve results and that the semi-supervised approach with single patch used for direct supervision. We only report results for
provides large boost, the best model as measured by the reward on validation set across several training runs. App. Fig, shows validation reward curves consistently improve during training for all random seeds.
DISCUSSION AND OUTLOOK
We introduced an end-to-end instance segmentation algorithm that can exploit non con-iffferentiale loss functions and high-level prior information. Our novel RL approach is based on the stateless actor-critic and predicts the full segmentation at every step, allowing us to assign rewards to all objects and reach stable convergence. The segmentation problem is formulated as graph partitioning; we design reward decomposition algorithm which maps object- and image-level rewards to sub-graphs localized supervision. Our experiments demonstrate good segmentation quality on synthetic and real data using only rule-based supervision without any object- or pixel-level labels, such as centers, boxes or masks. Furthermore, in case of full supervision, our method enables end-to-end instance segmentation with direct object-level reasoning, which will allow for cost-processig-.. -warre training of segmentation CNNs. the future, we plan to explore other tasks and reward functions and will further study the semi-supervised setup that showed very promising initial results.
Limitations Our method relies on superpixels which are fixed and not optimized jointly, so the upper bound on the performance defined by superpixel accuracy. We believe an extension to pixels possible and envision working this in the future, but the current setup will not scale the pixel level directly. Also, our method is limited to problems where consistent prior rules can be formulated for all instances. While this is the case for many applications in life science and medical imaging, not all object classes in the natural world can described in this way. Here, our method could contribute by complementing training annotations with rules, reducing the overall labelling load in semi-supervised setting, Finally, our approach requires non-trivial reward engineering as a trade-off' for not performing any annotations.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Abbas, Ahmed and Paul Swoboda (2021). "Combinatorial Optimization for Panoptic Segmentation: Fully Differentiable Approach". In: Advunces in Neural Information Processing Systems 34. Andres, Bjoern et al. (2012). "Globally optimal closed-surface segmentation for connectomics". In: European Conference on Computer Vision. Springer, pp. 778-791.
Araslanov, Nikita, Constantin Rothkopf, and Stefan Roth (2019). "Actor- Instance Segmentation". In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Bagnell, Drew and Andrew Ng (2006). "On Local Rewards and Scaling Distributed Reinforeement Learn ing". In: Advances in Neural Information Processing Systems. Ed. by Y. Weiss, B. Schölkopf, and J. Platt. Vol. 18. MIT Press. URL: https: pttp://cccccccccceeeeeoo neurips cc paper 2005 / file/ (1007719b66090 9b669a
02180771a
Bailoni, Alberto et al. (2019). "A generalized framework for agglomerative clustering of signed graphs applied to instance segmentation". In: arXiv preprint arXiv: 11713.
Beier, Thorsten et al. (2017). "Multicut brings automated neurite segmentation closer to human performance".
In: Nature methods 14.2, pp. 101-102.
Berman, Maxim, Amal Rannen Triki, and Matthew B Blaschko (2018). "The lovász-softmax loss: tractable surrogate for the optimization of the intersection-over-un the IEEE Conference on Computer Vision and Puttern Recognition, pp. 4413-4421
measure neural networks". In: Proceedings of
Bhide, Sourabh et al. (2020). Semi-Attomatic Generation of Tighi Binary Masks and Non-Convex Isosurfaces for Quantitative Analysis of 3D Biological Samples. arXiv: 22C01.1146 [eess. IV] Brabandere, Bert De. Davy Neven, and Luc Van Gool (2017). Semantic Instance Segmentation with Discrimi- native Loss Function. arXiv: 17708. C2551 [cs.cV]
Caicedo, Juan C et al. (2019). "Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl". In: Nature methods 16. pp. 1247-1253.
Chamier, Lucas von al. (Apr. 2021). "Democratising deep learning for microscopy with MeroCos In: Nature Communications.
Chen, Liang-Chich, Huiyu Wang, and Siyuan Qiao (2021). Scaling Wide Residual Networks for Panoptic Segmentation, arXiv: 2011.11675 cV.
Comaniciu, Dorin and Peter Meer (2002). "Mean shift: A robust approach toward feature space analysis". In:
IEEE Trunsactions on pattern analysis and machine intelligence 24.5. pp. 603-619.
De Brabandere, Bert, Davy Neven, and Lue Van Gool (2017). "Semantic instance segmentation with a discrimi- native loss function". In: arXiv preprint arXiv: (70::/008000 02551.
Bilaado-Gozzalo Ricard et al. (2014). "Snakes plane: A perfect snap for bioimage analysis". In: IEEE Signal Processing Magazine p 41-48.
Eban, Elad et al. (2017). "Scalable learning of non-decomposable objectives". In: Artificial intelligence and statistics. PMLR, pp. 832-840.
Falk, Thorsten et al. (2019). "U-Net: deep learning for cell counting, detection, and morphometry". In: Nature
methods 16.1, pp. 67-70.
Funke, Jan et al. (2018). "Large scale image segmentation with structured loss based deep learning for connec- tome reconstruction" " In: IEEE transactions on pattern analysis and machine intelligence 41.7, pp. 1669- 1680.
Gao, Naiyu et al. (2019). "Ssap: Single-shot instance segmentation with affinity pyramid". In: Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 642-651.
Gilmer, Justin et al. (2017). "Neural Message Passing for Quantum Chemistry". In: CoRR abs/1704.01212.
arXiv: arXiv: 1704. 01212 |RL:[attp://aae http:///axvvv..// www.v..ro//aba.... ott/abs//////oo.o./////..////bbr 704.01222
Grabocka, Josif, Randolf Scholz, and Lars Schmidt- Thieme (2019). "Learning surrogate losses". In: arXiv
preprint Xrxi: 00101995510000
Haamoja, Tuomas et al. (2018). "Soft Actor-Critic Algorithms and Applications". In: CoRR abs/1812.05905.
1xXiv[1812.. 05905 p /ttt:///axxvv http://xiw.o.oga/////////ooo 1tt.s/1111..9933333
Hassanein, Allam Shehata et al. (2015). "A Survey on Hough Transform, Theory, Techniques and Applications".
In: CoRR abs/ 552.022066026 60. arXiv: 1502. 02160 60 URL: ptttp:///axtt wervii.org//a ppg//bs/1550: 02.160
He, Kaiming al. (2017). "Mask r-cnn". In: Proceedings the IEEE international conference on computer
vision. pp.
Hu. Xiaoling et al. (2019). "Topology-Preservte
Deep Image Segmentation". In: Advances in Neural Information Processing Systems. Vol. 32. URL.: | / http://occcediieecoo ngs. neuri wwwpppppppppppccpcccpcc...ee.e /201 2d95666e266 64 9 wwf.ffc.e..0 7 5addb Patiaaaabp-appppaaee
Jain, Viren al. (Jan. 2011). "Learning to Agglomerate Superpixel Hicrarchics". In: Advances in Neural
Information Processing Systems 24.
Keriighan, Brian W and Shen Lin (1970). "An efficient heuristic procedure for partitioning graphs". In: The Bell
system technical journal 49.2. pp. 291-307.
Kervadee, Hoel et al. (2019). "Constrained-CNT losses for weakly supervised segmentation". In: Medical Image
Analysis 54, pp. 88 99. ISSN: 1361-8415. DOL:
modia..... .ww....1..0.
009
10

---

Under review as a conference paper at ICLR 2023
Lee, Kisuk al. (2017). Superhuman Accuracy on the SNEMI3D Connectomics Challenge. arXiv: 1736
[00120 [cs. CV] Maitin-Shepe -hittinSheppard, Jeremy B et al. (2016). "Combinatorial Energy Learning for Image Segmentation".
In: Advances in Neural Information Processing Systems. Ed. by D. Lee et al. Vol. 29. Curran As- proceedings neurips cc paper / 2016 / file 2ddeaae2000ddd6dddd-2---0
sociates, Inc. URL: https
918576449c40720377
McInnes, Leland and John Healy (2017). "Accelerated hierarchical density based clustering". In: 2017 IEEE
International Conference on Data Mining Workshops (ICDMW). IEEE, pp. 33-42.
Meilä, Marina (2003). "Comparing clusterings by the variation of information" In: Learning theory and kernel machines. Springer, pp. 173-187.
Neven, Davy et al. (2019). "Instance segmentation by jointly optimizing spatial embeddings and clustering
bandwidth". " In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8837-8845.
Oktay. Ozan et al. (2018). "Anatomically Constrained Neural Networks (ACNNs); Application to Cardiac Image Enhancement and Segmentation". In: IEEE Transactions on Medical Imaging 37.2, pp. 384-395. DOI:
10.1109/ 10111999MMM... 2C17. 2743464]
Osher, S. and N. Paragios (2007). Geometric Level Set Methods in Imaging, Vision, and Graphics Springer New
York. ISBN: 9780387218100
URL: htt::ttss:: /books..os///// . googlc. de /wwokboossee mid=-idddi- 2ww2rBaa... TWWARQBAA
Otsu, Nobuyuki (1979). "A threshold selection method from gray- -evel histograms" In: IEEE transactions on systems, man, and cybernetics 9.1, pp. 62-66.
Pape, Constantin al. (2019). "Leveraging domain knowledge to improve microscopy image segmentation with lifted multicuts". In: Frontiers in Computer Science 1, p. 6.
Pfau, David and Oriol Vinyals (2016). "Connecting Generative Adversarial Networks and Actor-Critic Methods".
In: CoRR a 610.01944 Xixiv: J1945 URL: ttt /arxivaivvvv..v////////aa///aiim org abs / 1 610 01943 Rand, William (1971). "Objective criteria for the evaluation of clustering methods". In: Journal of the
American Statistical association 66.336, pp. 846-850.
Ronneberger, Olaf, Philipp Fischer, and Thomas Brox (2015). "U-Net: Convolutional Networks for Biomedical
Image Segmentation". In: CoRR abs/1505.04 04597. arXiv: 505 04597
URL: /ttp::/arxtt org/
0es/15055.45599
Santos, Cicero Nogueira dos, Kahini Wadhawan, and Bowen Zhou (2017). Learning Loss Functions for Semi-
supervised Learning via Discriminative Adversarial Networks. ar Xiv: [177. [707.02198 02198 [cs.LG]
Schmidt, Uwe et al. (2018). "Cell detection with sar-convex polygons". In: International Conference on Medical Image Computing and Computer-Assisedd Intervention. Springer, pp. 265-273.
Shamir, Reuben R. et (2019). "Continuous Dice Coefficient: a Method Evaluating Probabilistic Seg- mentations". In: CoRR abs/1906. 1031. Xiv: 1 1906. 11031
URI.:
11031
Song, Jie et al. (2019). "End-to-end learning for graph decomposition". In: Proceedings of the IERF/CVF
International Conference on Computer Vision, pp. 10093-10102.
Song. Yang et al. (2016). "Training Deep Neural Networks via Direct Loss Minimization". In: International Conference on Machine Leurning. Stringer, Carsen et al. (2021). "Cellpose: a generalist algorithm for cellular segmentation". In: Nature Methods
18.1, pp. 100-100 Sunchag, Peter et (2017). DaloppDecompoiitionn Networks For Cooperative Multi-Agent Learning. arXiv:
1706. 1706.05296 95296 cs.
Sutton, Richard S. and Andrew G. Barto (2018). Reinforcement Learning: An Introduction. Second. The MIT
Press. URL: http: : http://nnooopp/ee/////o//oo eetemppeeeeiieeee deas net,
book 2nd. .ttll
Thompson, D' Arcy Wentworth (1992). On Growth and Form. Ed. by John TylerEditor Bonner. Canto. Cambridge University Press. DOL: (10. 1w1111119911/111
7325852
Wolf, Steffen et al. (2020). "The Mutex Watershed and its Objective: Efficient, Parameter-Free Graph Partition-
ing". In: IEEE transactions Wolny, Adrian, Lorenzo Cerrone, al. (2020). "Accurate and versatile 3D segmentation of plant tissues cellular resolution". In: Elife c57613.
pattern analysis and machine intelligence.
Wolny, Adrian, Qin Yu, et al. (2021). "Sparse Object-level Supervision for Instance Segmentation with Pixel Embeddings".. In: arXiv preprini arXiv:2103.1455 14572.
Xu, Huazhe et al. (2019).
Plannnnng----g Learning task-agnostic priors from interactions
and sparse rewards for zero-shot gemeralization". In: CoRR abs/1910.08 08133. arXiv: 1910. 08143 URL:
http: www.vvrrgrabboee wwwpp/ap//.....o.r////r/
11


Reviews:
[{"review_id": "z5_jf0dLZs0", "reviewer": "Reviewer_qRMW", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "- The paper formulates instance segmentation as a RL problem based on a stateless actor-critic setup, achieving end-to-end learning while exploiting prior knowledge on instance appearance instead of using ground truth instance supervision.\n- The paper introduces a strategy for spatial decomposition of rewards based on fixed-sized subgraphs to enable localized supervision from combinations of object- and image-level rules. \n- The paper demonstrates the feasibility of the approach on synthetic and real images and delivers excellent segmentations without ground truth supervision.\n", "summary_of_the_review": "The paper is quite novel but the contribution is fairly limited.\n", "strength_and_weaknesses": "Strength:\nThe paper is well-motivated and well-written.\nThe overall RL environment looks sound and reasonable.\nThe experiments show the feasibility of the method.\n\nWeakness:\nThe first paragraphs of the section 3 method, which, in my opinion, tries to describe the RL formulation of the problem, is hard to grasp overall, especially on how the image is segmented by graph partition.\nAlthough the experiments show the feasibility of the approach, the method is quite limited to segmentation problems with strong priors. Therefore, the potential of expanding the idea to more real-world datasets is quite unclear, thus, it makes the contribution limited.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "igEM75IRm1", "reviewer": "Reviewer_qqMa", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper formulates the instance segmentation problem as a graph partitioning problem, and uses reinforcement learning to predict the edge weights, which are the inputs to multicut solver for object segmentation. The authors introduce a strategy for spatial decomposition of rewards based on fixed-sized subgraphs to generate the object-to-sub-graph reward mapping to guide the action prediction. The experiments on toy and real data demonstrate that a good set of priors is sufficient to reach excellent performance without any direct object-level supervision.", "summary_of_the_review": "The paper use actor critic to predict the edge weights and then use graph partition for instance segmentation. Overall it is a good paper, but not ready for publication.", "strength_and_weaknesses": "Quality/Clarity: the paper is well written and the techniques presented are easy to follow. Its motivation to use reinforcement learning for instance segmentation is clear, such as the non-differentiable step of instance extraction and lack of annotated training data.\n\nOriginality/significance: the idea is interesting, and there are multiple strengths:  (1) superpixels and graph neural network feature exaction (2) sub-graph reward mapping. However, it is a little weird to add reinforcement learning here. Additionally, the authors claim that there is no state transition. Without the buzzword \"reinforcement learning\", I think we can achieve similar results using the designs (GNN+reward mapping) in this paper.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Hi_PlboSSe", "reviewer": "Reviewer_KFcT", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper explores the instance segmentation via actor-critic reinforcement learning. Instead of large amount of object-level supervision, the author formulate the instance segmentation as graph partitioning and predict the edge weights driven by the reward from object shape, position and size. Experiments on toy and real data demonstrate the effectiveness of the methods with a good set of priors. \nOverall the proposed algorithm provides a new view regarding prior knowledge embedding, which enable well trained model with less annotated labels. ", "summary_of_the_review": "This paper address the limited labels for instance segmentation thorough actor-critic reinforcement learning. It follows the standard RL framework and take the object shape, position and size as the prior. By assigning the reward for each actor, the optimal instance score is obtained. \nOverall the paper is well origanized but the experiments are not adquate to demonstrate the effectiveness of the algorithm. The reviewer suggest the author conduct more experiments on mote compilicated benchmark for instance segmentation.  ", "strength_and_weaknesses": "Strength:\n\n+Training model with non-differentiable factors. When dealing with limited samples, the reinforcement learning plays a key role and the author take the advantage of it.\n\n+The experiments is interesing and complete. They are conducted on both synthic and real images. \n\nWeakness:\n\n-The reason for utilizing the superpixel as the prior is not well explained. \n-The estimation to computatial cost and inference speed (e.g., FLOPs, FPS) are not presented in the experiments. \n\n-Comparisons to standard instance segmentation are missing. Although the algorithm has the advantage in limited samples, it doesn't provide enough avidence to demonstrate the compatitive scores when the training samples are adquated\n\n-Experiments on real dataset is too simple to indicates the potential ability. Authors should provide the results on more complicated medical images.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject