Original: Z ERO P IXEL D IRECTIONAL B OUNDARY BY V ECTOR T RANSFORM
Revised: Z ERO P IXEL D IRECTIONAL B OUNDARY BY V ECTOR T RANSFORM

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Boundaries or contours are among the primary visual cues used by human and computer vision systems.
Revised: Boundaries or contours are among the primary visual cues used by human and computer vision systems.

Original: One of the key problems in boundary detection is the loss formulation, which typically leads to class imbalance and, as a consequence, to thick boundaries which require non-differential post-processing steps to be thinned.
Revised: One of the key problems in boundary detection is the loss formulation, which typically leads to class imbalance and, as a consequence, to thick boundaries which require non-differential post-processing steps to be thinned.

Original: In this paper, we re-interpret boundaries as 1-D surfaces and formulate a one-to-one vector transform function that allows for training of boundary prediction completely avoiding the class imbalance issue.
Revised: In this paper, we re-interpret boundaries as 1-D surfaces and formulate a one-to-one vector transform function that allows for training of boundary prediction completely avoiding the class imbalance issue.

Original: Speciﬁcally, we deﬁne the boundary representation at any point as the unit vector pointing to the closest boundary surface.
Revised: Speciﬁcally, we deﬁne the boundary representation at any point as the unit vector pointing to the closest boundary surface.

Original: Our problem formulation leads to the estimation of direction as well as richer contextual information of the boundary, and, if desired, the availability of zero-pixel thin boundaries also at training time.
Revised: Our problem formulation leads to the estimation of direction as well as richer contextual information of the boundary, and, if desired, the availability of zero-pixel thin boundaries also at training time.

Original: Our method uses no hyper-parameter in the training loss and a ﬁxed stable hyper-parameter at inference.
Revised: Our method uses no hyper-parameter in the training loss and a ﬁxed stable hyper-parameter at inference.

Original: We provide theoretical justiﬁcation/discussions of the vector transform representation.
Revised: We provide theoretical justiﬁcation/discussions of the vector transform representation.

Original: We evaluate the proposed loss method using a standard architecture and show the excellent performance over other losses and representations on several datasets.
Revised: We evaluate the proposed loss method using a standard architecture and show the excellent performance over other losses and representations on several datasets.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Boundaries are considered to be one of the important interpretable visual cues that can describe both the low-level image characteristics as well as high-level semantics of the image.
Revised: Boundaries are considered to be one of the important interpretable visual cues that can describe both the low-level image characteristics as well as high-level semantics of the image.

Original: Human vision uses occluding contours and boundaries to interpret unseen or seen objects and classes.
Revised: Human vision uses occluding contours and boundaries to interpret unseen or seen objects and classes.

Original: Because boundaries are an important visual cue for image understanding, they are also exploited as priors in several vision tasks (Zhu et al., 2020;
Revised: Because boundaries are an important visual cue for image understanding, they are also exploited as priors in several vision tasks (Zhu et al., 2020;

Original: Kim et al., 2021; Hatamizadeh et al., 2019; Revaud et al., 2015; Cashman & Fitzgibbon, 2012).
Revised: Kim et al., 2021; Hatamizadeh et al., 2019; Revaud et al., 2015; Cashman & Fitzgibbon, 2012).

Original: Some key works on contours (Cootes et al., 2001; Matthews &
Revised: Some key works on contours (Cootes et al., 2001; Matthews &

Original: Baker, 2004; Kass et al., 1988) have greatly impacted early research in computer vision.
Revised: Baker, 2004; Kass et al., 1988) have greatly impacted early research in computer vision.

Original: Although the advent of deep learning and end-to-end learning has somewhat shifted the focus away from interpretable visual cues, boundary discovery still remains an important task in computer vision.
Revised: Although the advent of deep learning and end-to-end learning has somewhat shifted the focus away from interpretable visual cues, boundary discovery still remains an important task in computer vision.

Original: Supervised deep learning has greatly transformed problems such as object detection and segmentation (Redmon et al., 2016; Chen et al., 2017; Cheng et al., 2020) by redeﬁning the problem (Kirillov et al., 2019), using high-quality datasets (Cordts et al., 2016;
Revised: Supervised deep learning has greatly transformed problems such as object detection and segmentation (Redmon et al., 2016; Chen et al., 2017; Cheng et al., 2020) by redeﬁning the problem (Kirillov et al., 2019), using high-quality datasets (Cordts et al., 2016;

Original: Neuhold et al., 2017) and better network architectures (Cheng et al., 2020; 2021; Wang et al., 2021b).
Revised: Neuhold et al., 2017) and better network architectures (Cheng et al., 2020; 2021; Wang et al., 2021b).

Original: Boundary detection, however, has seen a rather modest share of such progress.
Revised: Boundary detection, however, has seen a rather modest share of such progress.

Original: Although, modern deeply learned methods (Xie & Tu, 2015; Liu et al., 2017; Maninis et al., 2017) provide better accuracy and the possibility to learn only the high-level boundaries, a particularly elusive goal in learned boundary detection has been the so-called crisp boundaries (Isola et al., 2014; Wang et al., 2018; Deng et al., 2018).
Revised: Although, modern deeply learned methods (Xie & Tu, 2015; Liu et al., 2017; Maninis et al., 2017) provide better accuracy and the possibility to learn only the high-level boundaries, a particularly elusive goal in learned boundary detection has been the so-called crisp boundaries (Isola et al., 2014; Wang et al., 2018; Deng et al., 2018).

Original: The formulation of boundary detection as a binary segmentation task naturally introduces class imbalance, which makes detecting pixel thin sharp boundaries extremely difﬁcult.
Revised: The formulation of boundary detection as a binary segmentation task naturally introduces class imbalance, which makes detecting pixel thin sharp boundaries extremely difﬁcult.

Original: Arguably a majority of recent methods in boundary detection are proposed in order to tackle the aforementioned issue.
Revised: Arguably a majority of recent methods in boundary detection are proposed in order to tackle the aforementioned issue.

Original: Many methods address the lack of ‘crispness’ by fusing high-resolution features with the middleand high-level features (Xie & Tu, 2015; Liu et al., 2017).
Revised: Many methods address the lack of ‘crispness’ by fusing high-resolution features with the middleand high-level features (Xie & Tu, 2015; Liu et al., 2017).

Original: Such a strategy has been successful in other dense prediction tasks (Ronneberger et al., 2015) as well.
Revised: Such a strategy has been successful in other dense prediction tasks (Ronneberger et al., 2015) as well.

Original: Others propose different loss functions (Kokkinos, 2016; Deng et al., 2018; Kervadec et al., 2019) to address class imbalance.
Revised: Others propose different loss functions (Kokkinos, 2016; Deng et al., 2018; Kervadec et al., 2019) to address class imbalance.

Original: Despite the improvements, we identify two issues regarding crisp boundary detection.
Revised: Despite the improvements, we identify two issues regarding crisp boundary detection.

Original: The ﬁrst is that the evaluation protocol (Martin et al., 2004) does not necessarily encourage crisp detection, as the quantiﬁcation is done after Non-Maximal Suppression (NMS).
Revised: The ﬁrst is that the evaluation protocol (Martin et al., 2004) does not necessarily encourage crisp detection, as the quantiﬁcation is done after Non-Maximal Suppression (NMS).

Original: Such an evaluation may be misleading when the network outputs need to be used at training time for other high-level tasks, e.g., segmentation (Kim et al., 2021).
Revised: Such an evaluation may be misleading when the network outputs need to be used at training time for other high-level tasks, e.g., segmentation (Kim et al., 2021).

Original: Second, the current losses (Kokkinos, 2016; Xie & Tu, 2015;
Revised: Second, the current losses (Kokkinos, 2016; Xie & Tu, 2015;

Original: Ma et al., 2021) push for edges as crisp as the ground-truth rather than as crisp as possible.
Revised: Ma et al., 2021) push for edges as crisp as the ground-truth rather than as crisp as possible.

Original: This is particularly harmful since many boundary detection datasets (Arbelaez et al., 2010; Silberman et al., 2012) contain ambiguous boundaries or inconsistently thick boundaries.
Revised: This is particularly harmful since many boundary detection datasets (Arbelaez et al., 2010; Silberman et al., 2012) contain ambiguous boundaries or inconsistently thick boundaries.

Original: In this paper, we take a different perspective on boundary detection.
Revised: In this paper, we take a different perspective on boundary detection.

Original: Boundaries are formed where visual features change, popularly referred to as the differential representation (Boykov et al., 2006;
Revised: Boundaries are formed where visual features change, popularly referred to as the differential representation (Boykov et al., 2006;

Original: Kervadec et al., 2019).
Revised: Kervadec et al., 2019).

Original: Such an assumption treats the boundaries as a set of 1-D surfaces embedded in a continuous 2D space, implying they do not have any thickness.
Revised: Such an assumption treats the boundaries as a set of 1-D surfaces embedded in a continuous 2D space, implying they do not have any thickness.

Original: Many previous energyminimization based approaches (Chan & Zhu, 2005; Chan & Vese, 2001; Paragios et al., 2002;
Revised: Many previous energyminimization based approaches (Chan & Zhu, 2005; Chan & Vese, 2001; Paragios et al., 2002;

Original: Boykov et al., 2006; Ma & Manjunath, 2000) and a few current methods (Kervadec et al., 2019) tackle boundaries in a similar way.
Revised: Boykov et al., 2006; Ma & Manjunath, 2000) and a few current methods (Kervadec et al., 2019) tackle boundaries in a similar way.

Original: Level-set methods (Chan & Zhu, 2005; Boykov et al., 2006;
Revised: Level-set methods (Chan & Zhu, 2005; Boykov et al., 2006;

Original: Ma & Manjunath, 2000) consider boundaries as the level-set of a continuous function of the image.
Revised: Ma & Manjunath, 2000) consider boundaries as the level-set of a continuous function of the image.

Original: Speciﬁcally, (Ma & Manjunath, 2000) deﬁnes the energy function related to the distance and direction of the boundary at each pixel and extracts the directional normals at the boundary using such an energy.
Revised: Speciﬁcally, (Ma & Manjunath, 2000) deﬁnes the energy function related to the distance and direction of the boundary at each pixel and extracts the directional normals at the boundary using such an energy.

Original: Inspired by such works and also recent works on 3D implicit functions (Tancik et al., 2020; Sitzmann et al., 2020; Mildenhall et al., 2020), we represent boundaries via a ﬁeld of unit vectors deﬁned at each pixel, pointing towards the closest boundary surface.
Revised: Inspired by such works and also recent works on 3D implicit functions (Tancik et al., 2020; Sitzmann et al., 2020; Mildenhall et al., 2020), we represent boundaries via a ﬁeld of unit vectors deﬁned at each pixel, pointing towards the closest boundary surface.

Original: The proposed vector ﬁeld representation naturally solves class imbalance.
Revised: The proposed vector ﬁeld representation naturally solves class imbalance.

Original: In distance transforms, vector ﬁelds are considered incomplete euclidean transforms (Osher & Sethian, 1988), equal to the Jacobian of the signed distance ﬁeld.
Revised: In distance transforms, vector ﬁelds are considered incomplete euclidean transforms (Osher & Sethian, 1988), equal to the Jacobian of the signed distance ﬁeld.

Original: The vector ﬁeld we use is in fact the Jacobian of the positive distance ﬁeld.
Revised: The vector ﬁeld we use is in fact the Jacobian of the positive distance ﬁeld.

Original: In contrast to distance ﬁelds, it provides high sensitivity at the boundaries and is easily localizable.
Revised: In contrast to distance ﬁelds, it provides high sensitivity at the boundaries and is easily localizable.

Original: We demonstrate the equivalence of the normal ﬁeld to the surface contour representation using the level set of the normal ﬁeld’s divergence, providing inﬁnitely sharp boundaries.
Revised: We demonstrate the equivalence of the normal ﬁeld to the surface contour representation using the level set of the normal ﬁeld’s divergence, providing inﬁnitely sharp boundaries.

Original: Owing to the zero-thickness, we refer to our result as the zero-pixel boundary.
Revised: Owing to the zero-thickness, we refer to our result as the zero-pixel boundary.

Original: Our method is virtually hyper-parameter free at training and test time, and can provide zero-pixel thick boundaries at training time.
Revised: Our method is virtually hyper-parameter free at training and test time, and can provide zero-pixel thick boundaries at training time.

Original: In order to evaluate the boundaries using the surface interpretation, we also advocate the use of surface distances including the average symmetric surface distance (assd) metric that is less prone to class imbalance and variable boundary thickness in the ground-truth (Kervadec et al., 2019).
Revised: In order to evaluate the boundaries using the surface interpretation, we also advocate the use of surface distances including the average symmetric surface distance (assd) metric that is less prone to class imbalance and variable boundary thickness in the ground-truth (Kervadec et al., 2019).

Original: Such metrics are very popular in biomedical image segmentation (Yeghiazaryan & Voiculescu, 2018).
Revised: Such metrics are very popular in biomedical image segmentation (Yeghiazaryan & Voiculescu, 2018).

Original: We show signiﬁcant improvements in all metrics using our boundary representation when compared to the various combinations of Dice (Dice, 1945) and cross-entropy losses in several large datasets.
Revised: We show signiﬁcant improvements in all metrics using our boundary representation when compared to the various combinations of Dice (Dice, 1945) and cross-entropy losses in several large datasets.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: There is a rich history of boundary detection in computer vision.
Revised: There is a rich history of boundary detection in computer vision.

Original: Previous work on boundaries showed a diversity of deﬁnitions and approaches.
Revised: Previous work on boundaries showed a diversity of deﬁnitions and approaches.

Original: We differentiate them based on two different interpretations of boundaries: i.e., i) a boundary is a separation between two or more image regions with different visual features and ii) a boundary is a thin group of pixels belonging to a speciﬁc class.
Revised: We differentiate them based on two different interpretations of boundaries: i.e., i) a boundary is a separation between two or more image regions with different visual features and ii) a boundary is a thin group of pixels belonging to a speciﬁc class.

Original: It should be noted that most modern methods fall under the second category.
Revised: It should be noted that most modern methods fall under the second category.

Original: Perhaps the most notable representatives of the ﬁrst strand are the energy-based segmentation methods (Chan & Vese, 2001; Comaniciu & Meer, 2002; Boykov et al., 2006; Grady, 2006; Ma & Manjunath, 2000).
Revised: Perhaps the most notable representatives of the ﬁrst strand are the energy-based segmentation methods (Chan & Vese, 2001; Comaniciu & Meer, 2002; Boykov et al., 2006; Grady, 2006; Ma & Manjunath, 2000).

Original: These methods relied on hand-crafted features and various optimization strategies to compute the low-level boundaries.
Revised: These methods relied on hand-crafted features and various optimization strategies to compute the low-level boundaries.

Original: In particular, Ma & Manjunath (2000) compute the normal vectors of the low-level boundaries from an energy function, without looking into an equivalent learnable representation.
Revised: In particular, Ma & Manjunath (2000) compute the normal vectors of the low-level boundaries from an energy function, without looking into an equivalent learnable representation.

Original: Graph-based segmentation methods (Shi & Malik, 2000; Felzenszwalb &
Revised: Graph-based segmentation methods (Shi & Malik, 2000; Felzenszwalb &

Original: Huttenlocher, 2004; Cheng et al., 2016) construct a graph from the image and cut the graph to obtain non-overlapping regions whose boundaries are viewed as image boundaries.
Revised: Huttenlocher, 2004; Cheng et al., 2016) construct a graph from the image and cut the graph to obtain non-overlapping regions whose boundaries are viewed as image boundaries.

Original: A few deep learning methods followed a similar approach (Wang et al., 2021a; Kervadec et al., 2019).
Revised: A few deep learning methods followed a similar approach (Wang et al., 2021a; Kervadec et al., 2019).

Original: Despite the advantage of the deﬁnition, current representations in this category are hard to adapt to generic boundaries and a compact multi-boundary representation with good performance remains lacking.
Revised: Despite the advantage of the deﬁnition, current representations in this category are hard to adapt to generic boundaries and a compact multi-boundary representation with good performance remains lacking.

Original: A larger corpus of work utilizes pixel-wise image features to decide whether pixels belong to a ‘boundary’ class.
Revised: A larger corpus of work utilizes pixel-wise image features to decide whether pixels belong to a ‘boundary’ class.

Original: They form our category ii) methods.
Revised: They form our category ii) methods.

Original: Early methods utilize various ﬁlter operators to detect discontinuities in image intensities or colors (Canny, 1986; Sobel, 1972).
Revised: Early methods utilize various ﬁlter operators to detect discontinuities in image intensities or colors (Canny, 1986; Sobel, 1972).

Original: Learning based methods substantially boost the edge detection performance by classifying handcrafted
Revised: Learning based methods substantially boost the edge detection performance by classifying handcrafted

Original: [Figure]
Revised: [Figure]

Original: features (Konishi et al., 2003; Martin et al., 2004; Arbelaez et al., 2010; Doll ´ ar & Zitnick, 2013;
Revised: features (Konishi et al., 2003; Martin et al., 2004; Arbelaez et al., 2010; Doll ´ ar & Zitnick, 2013;

Original: Hallman & Fowlkes, 2015).
Revised: Hallman & Fowlkes, 2015).

Original: Modern deep neural network (DNN) methods have further improved this ﬁeld by learning powerful feature representations, particularly high-level semantic information (Shen et al., 2015;
Revised: Modern deep neural network (DNN) methods have further improved this ﬁeld by learning powerful feature representations, particularly high-level semantic information (Shen et al., 2015;

Original: Bertasius et al., 2015; Xie & Tu, 2015; Liu et al., 2017; Wang et al., 2018; Deng et al., 2018;
Revised: Bertasius et al., 2015; Xie & Tu, 2015; Liu et al., 2017; Wang et al., 2018; Deng et al., 2018;

Original: He et al., 2019).
Revised: He et al., 2019).

Original: Yang et al.
Revised: Yang et al.

Original: (2016) leveraged the powerful deep features to detect only object boundaries.
Revised: (2016) leveraged the powerful deep features to detect only object boundaries.

Original: Others try to simultaneously detect edges and predict the semantic class of each edge point, so-called semantic edge detection (Hariharan et al., 2011; Yu et al., 2017;
Revised: Others try to simultaneously detect edges and predict the semantic class of each edge point, so-called semantic edge detection (Hariharan et al., 2011; Yu et al., 2017;

Original: Liu et al., 2018; Yu et al., 2018).
Revised: Liu et al., 2018; Yu et al., 2018).

Original: On the other hand, classifying pixels as boundary class introduces class imbalance during training.
Revised: On the other hand, classifying pixels as boundary class introduces class imbalance during training.

Original: A common counter-strategy is to use a weighted cross-entropy loss giving the non-boundary pixels a small weight and the boundary class a large weight (Xie & Tu, 2015;
Revised: A common counter-strategy is to use a weighted cross-entropy loss giving the non-boundary pixels a small weight and the boundary class a large weight (Xie & Tu, 2015;

Original: Liu et al., 2017; He et al., 2019).
Revised: Liu et al., 2017; He et al., 2019).

Original: Yet, despite an improvement over regular cross-entropy, it does not solve the problem.
Revised: Yet, despite an improvement over regular cross-entropy, it does not solve the problem.

Original: To thin the boundaries, Non-Maximal Suppression (NMS) is usually adopted.
Revised: To thin the boundaries, Non-Maximal Suppression (NMS) is usually adopted.

Original: Such methods may be harmful when directly integrated with higher-level tasks such as segmentation (Kim et al., 2021).
Revised: Such methods may be harmful when directly integrated with higher-level tasks such as segmentation (Kim et al., 2021).

Original: The Dice loss (Dice, 1945) was thus advocated to generate crisp boundaries before NMS (Deng et al., 2018), but it still produces several pixel thick boundaries and suffers more from missed predictions.
Revised: The Dice loss (Dice, 1945) was thus advocated to generate crisp boundaries before NMS (Deng et al., 2018), but it still produces several pixel thick boundaries and suffers more from missed predictions.

Original: Variations of the Dice loss (Shit et al., 2021) have been proposed to counter the missed detections.
Revised: Variations of the Dice loss (Shit et al., 2021) have been proposed to counter the missed detections.

Original: However, the right approach still depends on the downstream tasks (Zhu et al., 2020;
Revised: However, the right approach still depends on the downstream tasks (Zhu et al., 2020;

Original: Kim et al., 2021; Hatamizadeh et al., 2019; Shit et al., 2021) and in either case a careful selection of training as well as testing hyper-parameters is required.
Revised: Kim et al., 2021; Hatamizadeh et al., 2019; Shit et al., 2021) and in either case a careful selection of training as well as testing hyper-parameters is required.

Original: We provide an alternative approach, motivated by the class imbalance, while having no sensitive hyper-parameter.
Revised: We provide an alternative approach, motivated by the class imbalance, while having no sensitive hyper-parameter.

Original:  B OUNDARY T RANSFORM AND R EPRESENTATION
Revised:  B OUNDARY T RANSFORM AND R EPRESENTATION

Original: In this section, we ﬁrst discuss the surface representation of boundaries as a normal vector ﬁeld transform and prove its relevant properties.
Revised: In this section, we ﬁrst discuss the surface representation of boundaries as a normal vector ﬁeld transform and prove its relevant properties.

Original: Our boundary representation is inspired by recent work on implicit neural 3D surface representations (Park et al., 2019; Mescheder et al., 2019), energybased methods on edge detection (Ma & Manjunath, 2000; Boykov et al., 2006) and distance transforms (Osher & Sethian, 1988).
Revised: Our boundary representation is inspired by recent work on implicit neural 3D surface representations (Park et al., 2019; Mescheder et al., 2019), energybased methods on edge detection (Ma & Manjunath, 2000; Boykov et al., 2006) and distance transforms (Osher & Sethian, 1988).

Original: In 3D surface representations, a Signed Distance Function (SDF) (Osher & Sethian, 1988) or occupancy map (Mescheder et al., 2019) is used as representation.
Revised: In 3D surface representations, a Signed Distance Function (SDF) (Osher & Sethian, 1988) or occupancy map (Mescheder et al., 2019) is used as representation.

Original: We instead propose a unit vector ﬁeld from every point to the closest boundary.
Revised: We instead propose a unit vector ﬁeld from every point to the closest boundary.

Original: This choice is motivated by the high sensitivity and richer boundary context provided by the unit vector ﬁeld, as shown in our experimental results in § 5.
Revised: This choice is motivated by the high sensitivity and richer boundary context provided by the unit vector ﬁeld, as shown in our experimental results in § 5.

Original: Fig.
Revised: Fig.

Original: 1 shows our vector ﬁeld representation of the ground-truth, with predictions using a standard quiver plot on a sub-sampled set of points.
Revised: 1 shows our vector ﬁeld representation of the ground-truth, with predictions using a standard quiver plot on a sub-sampled set of points.

Original: We assume a continuous boundary image domain Ω ⊂ R 2 with the set of boundary points { x (cid:48) }
Revised: We assume a continuous boundary image domain Ω ⊂ R 2 with the set of boundary points { x (cid:48) }

Original: = Π ⊂ Ω .
Revised: = Π ⊂ Ω .

Original: Each point is denoted as a 2-vector x = ( x, y ) ∈ R 2 .
Revised: Each point is denoted as a 2-vector x = ( x, y ) ∈ R 2 .

Original: In order to encode boundary properties on the whole image, we compute a signed x and y distance ﬁeld separately and ﬁnally encode only the direction.
Revised: In order to encode boundary properties on the whole image, we compute a signed x and y distance ﬁeld separately and ﬁnally encode only the direction.

Original: The result is a unit vector ﬁeld that represents the boundary.
Revised: The result is a unit vector ﬁeld that represents the boundary.

Original: We can express our boundary representation by the following transform for any point x ∈ Ω :
Revised: We can express our boundary representation by the following transform for any point x ∈ Ω :

Original: [Equation]
Revised: [Equation]

Original: Equation (1) deﬁnes the transform as a function v ( x ) :
Revised: Equation (1) deﬁnes the transform as a function v ( x ) :

Original: Ω → R 2 going from the boundary Π to a ﬁeld representation.
Revised: Ω → R 2 going from the boundary Π to a ﬁeld representation.

Original: Here, d x and d y are operators that give the signed x and y difference, respectively.
Revised: Here, d x and d y are operators that give the signed x and y difference, respectively.

Original: Similarly, f x and f y are the x and y components of the ﬁeld f .
Revised: Similarly, f x and f y are the x and y components of the ﬁeld f .

Original: Note that we choose the ﬁeld vector arbitrarily among the two possible values at the boundary using the positive d x distance.
Revised: Note that we choose the ﬁeld vector arbitrarily among the two possible values at the boundary using the positive d x distance.

Original: We note the following properties of the vector ﬁeld v .
Revised: We note the following properties of the vector ﬁeld v .

Original: Property 3.1 The vector ﬁeld v ( x ) is equal to the unit normal ﬁeld at the boundary.
Revised: Property 3.1 The vector ﬁeld v ( x ) is equal to the unit normal ﬁeld at the boundary.

Original: Proof This is a well known result (Osher & Fedkiw, 2003) and can be proved easily (see equation (2.4) in the reference).
Revised: Proof This is a well known result (Osher & Fedkiw, 2003) and can be proved easily (see equation (2.4) in the reference).

Original: The fact that we forcefully choose one normal over its negative directional normal at the boundary points does not affect the statement.
Revised: The fact that we forcefully choose one normal over its negative directional normal at the boundary points does not affect the statement.

Original: Property 3.2 Given a vector ﬁeld representation v ( x ) of a boundary, one can obtain the binary boundary representation by considering the following transform:
Revised: Property 3.2 Given a vector ﬁeld representation v ( x ) of a boundary, one can obtain the binary boundary representation by considering the following transform:

Original: [Equation]
Revised: [Equation]

Original: The original boundary set Π can then be found by taking the zero level set of g ( x ) + 2 , i.e.,
Revised: The original boundary set Π can then be found by taking the zero level set of g ( x ) + 2 , i.e.,

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: L 0 ( g + 2) .
Revised: L 0 ( g + 2) .

Original: [Equation]
Revised: [Equation]

Original: Proof In the inﬁnitesimal neighborhood of the boundary points, using property 3.1, the vector ﬁeld is normal to the boundary, provided that good approximate normals can be obtained from equation (1).
Revised: Proof In the inﬁnitesimal neighborhood of the boundary points, using property 3.1, the vector ﬁeld is normal to the boundary, provided that good approximate normals can be obtained from equation (1).

Original: As the neighborhood size approaches zero, the tangential vector components approach zero around a point for a continuous boundary segment.
Revised: As the neighborhood size approaches zero, the tangential vector components approach zero around a point for a continuous boundary segment.

Original: Thus, around such an inﬁnitesimal neighborhood, the normal ﬁelds pointing in opposite direction will subtract perfectly, creating a divergence ﬂow of -2 and around 0 or positive away from boundaries.
Revised: Thus, around such an inﬁnitesimal neighborhood, the normal ﬁelds pointing in opposite direction will subtract perfectly, creating a divergence ﬂow of -2 and around 0 or positive away from boundaries.

Original: Strictly speaking the result holds only for piece-wise smooth surfaces (Osher & Fedkiw, 2003), with lower than -2 divergence possible at discontinuous surface points.
Revised: Strictly speaking the result holds only for piece-wise smooth surfaces (Osher & Fedkiw, 2003), with lower than -2 divergence possible at discontinuous surface points.

Original: Property 3.

Revised: Property 3.

Original: The relation is one-to-one between the binary boundary representation and the proposed vector ﬁeld representation in a continuous domain.
Revised: The relation is one-to-one between the binary boundary representation and the proposed vector ﬁeld representation in a continuous domain.

Original: Proof This property is the result of equation (1), for the forward transform and equation (3) for the inverse transform, providing a one-to-one relation.
Revised: Proof This property is the result of equation (1), for the forward transform and equation (3) for the inverse transform, providing a one-to-one relation.

Original: Note that the vector ﬁeld transform as deﬁned in equation (1) has to correct for two different kinds of indeterminate states.
Revised: Note that the vector ﬁeld transform as deﬁned in equation (1) has to correct for two different kinds of indeterminate states.

Original: The ﬁrst is on the boundary, that is solved by using the right hand limit so that one of the two opposite directions is chosen consistently.
Revised: The ﬁrst is on the boundary, that is solved by using the right hand limit so that one of the two opposite directions is chosen consistently.

Original: The second is when the inﬁmum operation in equation (1) produces two or more closest points, corrected by choosing any one of the points for the inﬁmum.
Revised: The second is when the inﬁmum operation in equation (1) produces two or more closest points, corrected by choosing any one of the points for the inﬁmum.

Original: The vector ﬁelds around such points ﬂip directions creating a positive divergence as shown in Fig.
Revised: The vector ﬁelds around such points ﬂip directions creating a positive divergence as shown in Fig.

Original: More discussions are provided in §5 about the latter, which are in fact helpful for deciding superpixel centers.
Revised: More discussions are provided in §5 about the latter, which are in fact helpful for deciding superpixel centers.

Original: The above properties and their proofs are crucial for the validity of the proposed boundary representation and also to go from one representation to another for inference and visualization.
Revised: The above properties and their proofs are crucial for the validity of the proposed boundary representation and also to go from one representation to another for inference and visualization.

Original: Vector Transform and the Distance Transform.
Revised: Vector Transform and the Distance Transform.

Original: In essence, the normalized vector ﬁeld proposed in equation (1) is another representation of the distance transform.
Revised: In essence, the normalized vector ﬁeld proposed in equation (1) is another representation of the distance transform.

Original: Let φ ( x ) ∈ R + deﬁne the distance transform, then the vector ﬁeld v ( x ) in equation (1) can be obtained by the following partial derivatives (Osher & Sethian, 1988; Osher & Fedkiw, 2003):
Revised: Let φ ( x ) ∈ R + deﬁne the distance transform, then the vector ﬁeld v ( x ) in equation (1) can be obtained by the following partial derivatives (Osher & Sethian, 1988; Osher & Fedkiw, 2003):

Original: [Equation]
Revised: [Equation]

Original: One can optimize a given network by minimizing the loss on the distance transform (DT) or SDF (Dapogny & Frey, 2012;
Revised: One can optimize a given network by minimizing the loss on the distance transform (DT) or SDF (Dapogny & Frey, 2012;

Original: Caliva et al., 2019; Park et al., 2019) instead of using the loss on the normalized vector ﬁeld.
Revised: Caliva et al., 2019; Park et al., 2019) instead of using the loss on the normalized vector ﬁeld.

Original: Compared to the binary mask, the Vector transform (VT), DT and SDF have an added advantage that they are sensitive to small topological changes.
Revised: Compared to the binary mask, the Vector transform (VT), DT and SDF have an added advantage that they are sensitive to small topological changes.

Original: SDF on the other hand, does not support overlapping and open surfaces and is not easily adaptable to the image boundary problem.
Revised: SDF on the other hand, does not support overlapping and open surfaces and is not easily adaptable to the image boundary problem.

Original: However, there are several reasons which make DT unsuitable for learning boundaries.
Revised: However, there are several reasons which make DT unsuitable for learning boundaries.

Original: During training, when the distance ﬁeld is close to 0 i.e., around the boundaries, any loss on the
Revised: During training, when the distance ﬁeld is close to 0 i.e., around the boundaries, any loss on the

Original: DT or SDF loses importance.
Revised: DT or SDF loses importance.

Original: Apart from the convergence problems of DT with gradient descent
Revised: Apart from the convergence problems of DT with gradient descent

Original: Osher & Fedkiw (2003), DT is also hard to localize by thresholding under noise compared to the
Revised: Osher & Fedkiw (2003), DT is also hard to localize by thresholding under noise compared to the

Original: SDF and VT.
Revised: SDF and VT.

Original: In SDF, localizing the surface amounts to ﬁnding the zero crossings and in VT, the divergence measure in equation (2) provides an extremely sharp contrast making equation (3) trivial to solve.
Revised: In SDF, localizing the surface amounts to ﬁnding the zero crossings and in VT, the divergence measure in equation (2) provides an extremely sharp contrast making equation (3) trivial to solve.

Original: These differences in the thresholding problem can be seen in Fig.
Revised: These differences in the thresholding problem can be seen in Fig.

Original: 5 in Appendix B and the experimental results.
Revised: 5 in Appendix B and the experimental results.

Original: Additionally, despite reducing the class imbalance compared to binary boundary prediction, DT has an implicit bias to the weighted average of its typical range.
Revised: Additionally, despite reducing the class imbalance compared to binary boundary prediction, DT has an implicit bias to the weighted average of its typical range.

Original: On the other hand, a normalized vector ﬁeld from VT equation (1) is sensitive to the topology similar to a distance ﬁeld while also being localizable and sensitive at the boundaries, as shown in Fig.
Revised: On the other hand, a normalized vector ﬁeld from VT equation (1) is sensitive to the topology similar to a distance ﬁeld while also being localizable and sensitive at the boundaries, as shown in Fig.

Original:  B OUNDARY D ETECTION WITH THE V ECTOR F IELD
Revised:  B OUNDARY D ETECTION WITH THE V ECTOR F IELD

Original: In this section we provide the details for the construction of the boundary detection method using the representation proposed in §3.
Revised: In this section we provide the details for the construction of the boundary detection method using the representation proposed in §3.

Original: 4. N ETWORK A RCHITECTURE
Revised: 4. N ETWORK A RCHITECTURE

Original: Most convolutional architectures (Liu et al., 2017; Xie & Tu, 2015) for boundary detection take advantage of both the low-level high resolution features and the deep high-level features, using several fusion strategies.
Revised: Most convolutional architectures (Liu et al., 2017; Xie & Tu, 2015) for boundary detection take advantage of both the low-level high resolution features and the deep high-level features, using several fusion strategies.

Original: We choose a similar network architecture HR-Net (Wang et al., 2020) which was proposed for segmentation and object detection tasks in high resolution images.
Revised: We choose a similar network architecture HR-Net (Wang et al., 2020) which was proposed for segmentation and object detection tasks in high resolution images.

Original: We further enrich the high resolution representation provided by HR-Net with an additional skip connection at ×downsampling level from the encoder to the decoder.
Revised: We further enrich the high resolution representation provided by HR-Net with an additional skip connection at ×downsampling level from the encoder to the decoder.

Original: This helps retrieve very high resolution details that are necessary in the boundary prediction task.
Revised: This helps retrieve very high resolution details that are necessary in the boundary prediction task.

Original: The output of the HR-Net is ﬁrst bilinearly upsampled and fused with the skip connection.
Revised: The output of the HR-Net is ﬁrst bilinearly upsampled and fused with the skip connection.

Original: Inspired by Deng et al.
Revised: Inspired by Deng et al.

Original: (2018), the high resolution skip connection goes through a ResNeXT module (Xie et al., 2017) before being fused with the decoder signal with a pixel-wise fully connected layer.
Revised: (2018), the high resolution skip connection goes through a ResNeXT module (Xie et al., 2017) before being fused with the decoder signal with a pixel-wise fully connected layer.

Original: Finally, the output goes through a convolutional layer and is further upsampled to formulate a prediction at the same resolution as the input.
Revised: Finally, the output goes through a convolutional layer and is further upsampled to formulate a prediction at the same resolution as the input.

Revised: For more details refer to Appendix §F.

Original: This architecture allows us to test our method as well as traditional, pixel-wise classiﬁcation methods, without giving an unfair advantage to one over the other.
Revised: This architecture allows us to test our method as well as traditional, pixel-wise classiﬁcation methods, without giving an unfair advantage to one over the other.

Original: Fig. 2 shows the simple setup of our complete pipeline with training and inference.
Revised: Fig.shows the simple setup of our complete pipeline with training and inference.

Original: The output of our network is a two channel image which contains, respectively, the x-component ˆ v x and the y-component ˆ v y of the ﬁeld prediction corresponding to equation (1).
Revised: The output of our network is a two channel image which contains, respectively, the x-component ˆ v x and the y-component ˆ v y of the ﬁeld prediction corresponding to equation (1).

Original: We denote the transform of equation (1) as Vector Transform (VT), upon which we build our method.
Revised: We denote the transform of equation (1) as Vector Transform (VT), upon which we build our method.

Original: Although it is possible to use a single angle prediction and representation as in θ ∈ [ − π, π ] , we choose the x and y component representation because it avoids discontinuities in its value as the vector changes.
Revised: Although it is possible to use a single angle prediction and representation as in θ ∈ [ − π, π ] , we choose the x and y component representation because it avoids discontinuities in its value as the vector changes.

Original: To constrain the network to output predictions in the [ − 1 , 1] range typical of the sine and cosine functions, a hyperbolic tangent activation ( tanh ) is applied to the output of the network.
Revised: To constrain the network to output predictions in the [ − 1 , 1] range typical of the sine and cosine functions, a hyperbolic tangent activation ( tanh ) is applied to the output of the network.

Original: Training loss.
Revised: Training loss.

Original: To train the x and y components of VT prediction, the mean-squared error (MSE) loss is used, which can be formulated as
Revised: To train the x and y components of VT prediction, the mean-squared error (MSE) loss is used, which can be formulated as

Original: [Equation]
Revised: [Equation]

Original: Unlike many other methods, there exists no hyper-parameters on the loss formulation.
Revised: Unlike many other methods, there exists no hyper-parameters on the loss formulation.

Original: 4. I NFERENCE
Revised: 4. I NFERENCE

Original: At inference time, we predict the boundary VT densely on the image.
Revised: At inference time, we predict the boundary VT densely on the image.

Original: It is entirely possible that for some downstream tasks, such a prediction may already provide the required endpoint.
Revised: It is entirely possible that for some downstream tasks, such a prediction may already provide the required endpoint.

Original: However, to obtain the surface representation of the boundary, we need to use the properties listed in §3.
Revised: However, to obtain the surface representation of the boundary, we need to use the properties listed in §3.

Original: [Figure]
Revised: [Figure]

Original: In particular, property 3.2 provides a differential means to obtain a zero-thickness boundary on a continuous domain.
Revised: In particular, property 3.2 provides a differential means to obtain a zero-thickness boundary on a continuous domain.

Original: In the discrete space, the same principle can be used to derive the boundary as the inter-pixel points, which we refer to as the zero-pixel boundary.
Revised: In the discrete space, the same principle can be used to derive the boundary as the inter-pixel points, which we refer to as the zero-pixel boundary.

Original: At this stage, at least two different approaches can be used to obtain the inter-pixel separation.
Revised: At this stage, at least two different approaches can be used to obtain the inter-pixel separation.

Original: One can be formulated by using 2 × 1 or 2 × 2 kernels to extract the divergence, which provides the necessary derivatives at the pixel boundaries.
Revised: One can be formulated by using 2 × 1 or 2 × 2 kernels to extract the divergence, which provides the necessary derivatives at the pixel boundaries.

Original: We describe in detail a second and simpler approach which uses a support image ˜
Revised: We describe in detail a second and simpler approach which uses a support image ˜

Original: I of twice the resolution, to extract boundaries as a single pixel detection.
Revised: I of twice the resolution, to extract boundaries as a single pixel detection.

Original: We deﬁne an operator
Revised: We deﬁne an operator

Original: Z ( I ) =
Revised: Z ( I ) =

Original: ˜ I , which takes in any image I and provides the image ˜
Revised: ˜ I , which takes in any image I and provides the image ˜

Original: I of twice the resolution according to the following rule.
Revised: I of twice the resolution according to the following rule.

Original: [Equation]
Revised: [Equation]

Original: Equation (6) is a simple copy and concatenate operation.
Revised: Equation (6) is a simple copy and concatenate operation.

Original: Using the operator Z on the predicted VT ﬁeld ˆ v , we obtain ˜ v =
Revised: Using the operator Z on the predicted VT ﬁeld ˆ v , we obtain ˜ v =

Original: Z (ˆ v ) .
Revised: Z (ˆ v ) .

Original: We then compute the divergence using standard Sobel ﬁlters as:
Revised: We then compute the divergence using standard Sobel ﬁlters as:

Original: [Equation]
Revised: [Equation]

Original: The image with boundaries I b can then be obtained inverting the divergence of the VT prediction image in equation (7), subtracting 1 from it and applying ReLU activation (Nair & Hinton, 2010):
Revised: The image with boundaries I b can then be obtained inverting the divergence of the VT prediction image in equation (7), subtracting 1 from it and applying ReLU activation (Nair & Hinton, 2010):

Original: [Equation]
Revised: [Equation]

Original: The resulting image will have non-zero values only on the predicted boundary surface, with boundary pixels having values around 1 .
Revised: The resulting image will have non-zero values only on the predicted boundary surface, with boundary pixels having values around 1 .

Original: In practice we obtain small deviations of about 0 .
Revised: In practice we obtain small deviations of about 0 .

Original: 1 around the expected value of 1 due to imperfections in the ﬁeld predictions.
Revised: 1 around the expected value of 1 due to imperfections in the ﬁeld predictions.

Original: In the support image structure, it can be observed that only pixels not belonging to the original image can be classiﬁed as boundaries as they are the only ones for which the divergence can have a value different from zero.
Revised: In the support image structure, it can be observed that only pixels not belonging to the original image can be classiﬁed as boundaries as they are the only ones for which the divergence can have a value different from zero.

Original: Note that the above divergence deﬁnition using the support image is only required for zero-thickness boundary, we may prefer Fig.
Revised: Note that the above divergence deﬁnition using the support image is only required for zero-thickness boundary, we may prefer Fig.

Original: 2 for a standard inference process.
Revised: 2 for a standard inference process.

Original: In order to evaluate the zero-pixel boundary with traditional metrics, and visualize, boundary pixels need to be represented on the original image.
Revised: In order to evaluate the zero-pixel boundary with traditional metrics, and visualize, boundary pixels need to be represented on the original image.

Original: We do so, by copying the values of boundary pixels in the support image to the corresponding neighboring pixels belonging to the original image and averaging the values if necessary; all the other values are set to zero.
Revised: We do so, by copying the values of boundary pixels in the support image to the corresponding neighboring pixels belonging to the original image and averaging the values if necessary; all the other values are set to zero.

Original: This leads to two pixel boundaries with the added property of being differentiable, which may be particularly useful when integrating the boundary detection task in a larger deep network.
Revised: This leads to two pixel boundaries with the added property of being differentiable, which may be particularly useful when integrating the boundary detection task in a larger deep network.

Original: When evaluating the method with the surface distance metrics - which are discussed in §4.3 - the resulting image is thresholded to obtain a binary boundary image.
Revised: When evaluating the method with the surface distance metrics - which are discussed in §4.3 - the resulting image is thresholded to obtain a binary boundary image.

Original: More speciﬁcally, all the positive pixels are considered to belong to a boundary and this remains ﬁxed throughout the experiments.
Revised: More speciﬁcally, all the positive pixels are considered to belong to a boundary and this remains ﬁxed throughout the experiments.

Original: An important aspect of VT is that it provides directional information on each pixel, including those pixels which are associated to the boundaries.
Revised: An important aspect of VT is that it provides directional information on each pixel, including those pixels which are associated to the boundaries.

Original: Directional boundaries have been previously explored with applications and advantages (Maninis et al., 2017).
Revised: Directional boundaries have been previously explored with applications and advantages (Maninis et al., 2017).

Original: We provide two key examples.
Revised: We provide two key examples.

Original: In the ﬁrst application, we detect boundary pixels which are proposal candidates for straight lines in a differentiable way.
Revised: In the ﬁrst application, we detect boundary pixels which are proposal candidates for straight lines in a differentiable way.

Original: In the second, we detect superpixels by grouping pixels inside a convex boundary.
Revised: In the second, we detect superpixels by grouping pixels inside a convex boundary.

Original: These related tasks are discussed more in depth with some qualitative results in the appendices.
Revised: These related tasks are discussed more in depth with some qualitative results in the appendices.

Original: 4. M ETRICS
Revised: 4. M ETRICS

Original: The standard measures for boundary evaluation (Martin et al., 2004) are ﬁxed contour threshold (ODS) and per-image
Revised: The standard measures for boundary evaluation (Martin et al., 2004) are ﬁxed contour threshold (ODS) and per-image

Original: best threshold (OIS).
Revised: best threshold (OIS).

Original: With these approaches it is possible to compute the recall (R), the precision (P) and the F score.
Revised: With these approaches it is possible to compute the recall (R), the precision (P) and the F score.

Original: These are the metrics traditionally applied using the approach proposed by Martin et al.
Revised: These are the metrics traditionally applied using the approach proposed by Martin et al.

Original: (2004) to determine the true positives and false negatives.
Revised: (2004) to determine the true positives and false negatives.

Original: It proceeds by creating a one-to-one correspondence between the ground-truth boundary pixels and the predicted boundary pixels.
Revised: It proceeds by creating a one-to-one correspondence between the ground-truth boundary pixels and the predicted boundary pixels.

Original: In this context, each pixel without a correspondence within a ﬁxed threshold distance is considered either a false positive or a false negative.
Revised: In this context, each pixel without a correspondence within a ﬁxed threshold distance is considered either a false positive or a false negative.

Original: This approach, however, suffers from a few drawbacks:
Revised: This approach, however, suffers from a few drawbacks:

Original: • It is extremely sensitive in differences of thickness between the prediction and the ground-truth, as the one-to-one correspondence will not be satisﬁed resulting in a large number of false positives or false negatives without regard for the actual boundary quality.
Revised: • It is extremely sensitive in differences of thickness between the prediction and the ground-truth, as the one-to-one correspondence will not be satisﬁed resulting in a large number of false positives or false negatives without regard for the actual boundary quality.

Original: • ODS and OIS optimize the threshold hyper-parameter on the test set.
Revised: • ODS and OIS optimize the threshold hyper-parameter on the test set.

Original: This may lead to an unfair evaluation of the quality of the detected boundaries.
Revised: This may lead to an unfair evaluation of the quality of the detected boundaries.

Original: • As it is based on pixel correspondences, it cannot be directly applied to zero pixel thin boundaries.
Revised: • As it is based on pixel correspondences, it cannot be directly applied to zero pixel thin boundaries.

Original: To overcome these drawbacks and have a metric that can be applied to the zero pixel surfaces as well, we propose to use the average surface distances (asd), more favored in the medical imaging community (Yeghiazaryan & Voiculescu, 2018).
Revised: To overcome these drawbacks and have a metric that can be applied to the zero pixel surfaces as well, we propose to use the average surface distances (asd), more favored in the medical imaging community (Yeghiazaryan & Voiculescu, 2018).

Original: For every boundary pixel or segment in the prediction, it is the distance to the closest ground truth boundary surface point - either a pixel or a segment.
Revised: For every boundary pixel or segment in the prediction, it is the distance to the closest ground truth boundary surface point - either a pixel or a segment.

Original: The same can be done starting from the ground truth pixels or segments.
Revised: The same can be done starting from the ground truth pixels or segments.

Original: The distance from the prediction to the ground truth boundary ( asd P ) is representative for the quality of the prediction - a precision-like term - while the distance from the ground truth to the prediction ( asd R ), like recall, shows how effectively all the boundaries are detected.
Revised: The distance from the prediction to the ground truth boundary ( asd P ) is representative for the quality of the prediction - a precision-like term - while the distance from the ground truth to the prediction ( asd R ), like recall, shows how effectively all the boundaries are detected.

Original: The two scores can be averaged to obtain a single metric, the average symmetric surface distance ( assd ).
Revised: The two scores can be averaged to obtain a single metric, the average symmetric surface distance ( assd ).

Original: Given that no one-to-one correspondences between pixels are computed, asd P , asd R and assd are less sensitive to the boundary thickness and more inﬂuenced by the overall quality of the prediction, compared to the previously deﬁned metrics.
Revised: Given that no one-to-one correspondences between pixels are computed, asd P , asd R and assd are less sensitive to the boundary thickness and more inﬂuenced by the overall quality of the prediction, compared to the previously deﬁned metrics.

Original: However, one drawback of the surface distance metrics is that it has high sensitivity to isolated false detections or unmatched ground-truth boundary pixels.
Revised: However, one drawback of the surface distance metrics is that it has high sensitivity to isolated false detections or unmatched ground-truth boundary pixels.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We compare the proposed method on Cityscapes (Cordts et al., 2016), Mapillary Vistas (Neuhold et al., 2017) and Synthia (Ros et al., 2016), three datasets providing high quality instance and semantic boundaries.
Revised: We compare the proposed method on Cityscapes (Cordts et al., 2016), Mapillary Vistas (Neuhold et al., 2017) and Synthia (Ros et al., 2016), three datasets providing high quality instance and semantic boundaries.

Original: Despite the inconsistent thickness of annotated boundaries, we also compare our method on BSDS500 Arbelaez et al.
Revised: Despite the inconsistent thickness of annotated boundaries, we also compare our method on BSDS500 Arbelaez et al.

Original: The dataset contains a training size of just 200 images as well as relatively low image resolution.
Revised: The dataset contains a training size of just 200 images as well as relatively low image resolution.

Original: For this evaluation, every method is trained on the
Revised: For this evaluation, every method is trained on the

Original: BSDS500 training set, using the network pretrained on Mapillary Vistas.
Revised: BSDS500 training set, using the network pretrained on Mapillary Vistas.

Original: We compare VT against three different losses in binary representation; the dice loss (DL) (Dice, 1945), a weighted combination of Dice loss and cross-entropy (DCL) (Deng et al., 2018) and the weighted cross-entropy (WCL) (Xie & Tu, 2015).
Revised: We compare VT against three different losses in binary representation; the dice loss (DL) (Dice, 1945), a weighted combination of Dice loss and cross-entropy (DCL) (Deng et al., 2018) and the weighted cross-entropy (WCL) (Xie & Tu, 2015).

Original: As part of the ablation, we further compare our method against the Distance Transform (DT) representation of boundaries which predicts a ﬁeld of distances to the closest boundary.
Revised: As part of the ablation, we further compare our method against the Distance Transform (DT) representation of boundaries which predicts a ﬁeld of distances to the closest boundary.

Original: This is trained with a simple L1 loss between the ground truth d gt and the predicted distance ˆ d .
Revised: This is trained with a simple L1 loss between the ground truth d gt and the predicted distance ˆ d .

Original: Each representation and training loss is computed using the same network architecture and optimization.
Revised: Each representation and training loss is computed using the same network architecture and optimization.

Original: Evaluation Metrics.
Revised: Evaluation Metrics.

Original: We evaluate each method based on the traditionally used R, P and F score in the ODS and OIS setups as well as on asd R , asd P , and assd scores.
Revised: We evaluate each method based on the traditionally used R, P and F score in the ODS and OIS setups as well as on asd R , asd P , and assd scores.

Original: For the traditional metrics for BSDS500 dataset evaluation (Arbelaez et al., 2010), we use an error tolerance of 0 .
Revised: For the traditional metrics for BSDS500 dataset evaluation (Arbelaez et al., 2010), we use an error tolerance of 0 .

Original: 0025 of the diagonal length of the image.
Revised: 0025 of the diagonal length of the image.

Original: This is lower than what is commonly used on the BSDS500 dataset, to account for the larger image sizes.
Revised: This is lower than what is commonly used on the BSDS500 dataset, to account for the larger image sizes.

Original: To compute the surface distance metrics ( asd R , asd P , and assd ), each boundary representation is converted to a binary form with a thresholding operation.
Revised: To compute the surface distance metrics ( asd R , asd P , and assd ), each boundary representation is converted to a binary form with a thresholding operation.

Original: For the DL, DCL, WCL, and DT models, the threshold is ﬁxed using a selected validation set for each dataset.
Revised: For the DL, DCL, WCL, and DT models, the threshold is ﬁxed using a selected validation set for each dataset.

Original: For VT, instead, it is ﬁxed for each test to the value of − 1 of the divergence image, the same value added during inference before applying ReLU §4.2; points with lower divergence are classiﬁed as boundaries and the others as non-boundaries.
Revised: For VT, instead, it is ﬁxed for each test to the value of − 1 of the divergence image, the same value added during inference before applying ReLU §4.2; points with lower divergence are classiﬁed as boundaries and the others as non-boundaries.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 5. M ETRIC A NALYSIS
Revised: 5. M ETRIC A NALYSIS

Original: We ﬁrst show an analysis to support the use of the surface distance metrics (Yeghiazaryan &
Revised: We ﬁrst show an analysis to support the use of the surface distance metrics (Yeghiazaryan &

Original: Voiculescu, 2018), asd R , asd P , and assd over R, P
Revised: Voiculescu, 2018), asd R , asd P , and assd over R, P

Original: [Table]
Revised: [Table]

Original: and F score in ODS and OIS conditions.
Revised: and F score in ODS and OIS conditions.

Original: We use the VT method results on the Cityscapes dataset, where boundaries have a constant 2-pixel thickness.
Revised: We use the VT method results on the Cityscapes dataset, where boundaries have a constant 2-pixel thickness.

Original: To show the sensitivity of the metrics to different thicknesses in the prediction and ground truth, we compare the original prediction (OP) with the scores achieved when doubling the thickness of the prediction (TP) or of the ground truth (TG).
Revised: To show the sensitivity of the metrics to different thicknesses in the prediction and ground truth, we compare the original prediction (OP) with the scores achieved when doubling the thickness of the prediction (TP) or of the ground truth (TG).

Original: When computing the R, P, and F score, we use two experimental setups: (i) changing the used prediction threshold in an ODS and OIS fashion and (ii) and keeping it ﬁxed to the same value t 0 used in OP.
Revised: When computing the R, P, and F score, we use two experimental setups: (i) changing the used prediction threshold in an ODS and OIS fashion and (ii) and keeping it ﬁxed to the same value t 0 used in OP.

Original: For the surface distance metrics, the threshold is always ﬁxed as in every other experiment.
Revised: For the surface distance metrics, the threshold is always ﬁxed as in every other experiment.

Original: From the results in Table 1, it is clear that the R, P and F scores are strongly dependent on the relative thickness of ground truth and prediction with variations in the F score of over 27% versus only a 7% change in assd .
Revised: From the results in Table 1, it is clear that the R, P and F scores are strongly dependent on the relative thickness of ground truth and prediction with variations in the F score of over 27% versus only a 7% change in assd .

Original: The small inﬂuence of thickness shows that the surface distance metrics are suited to evaluate predictions without post-processing.
Revised: The small inﬂuence of thickness shows that the surface distance metrics are suited to evaluate predictions without post-processing.

Original: Furthermore, the evaluation also shows that the metric does not provide an advantage to the proposed method for its thin prediction.
Revised: Furthermore, the evaluation also shows that the metric does not provide an advantage to the proposed method for its thin prediction.

Original: 5. R EPRESENTATION P ERFORMANCE C OMPARISON
Revised: 5. R EPRESENTATION P ERFORMANCE C OMPARISON

Original: In this section, we show the performance of each representation on the four datasets considered.
Revised: In this section, we show the performance of each representation on the four datasets considered.

Original: We do not apply non-maximum suppression so as to keep similar post-processing conditions throughout all methods and to evaluate the methods under the same conditions, so they could be integrated in other tasks §1.
Revised: We do not apply non-maximum suppression so as to keep similar post-processing conditions throughout all methods and to evaluate the methods under the same conditions, so they could be integrated in other tasks §1.

Original: On Cityscapes, Synthia and Mapillary Vistas, our method consistently outperforms the other boundary representations in terms of assd , the metric less affected by the prediction thickness.
Revised: On Cityscapes, Synthia and Mapillary Vistas, our method consistently outperforms the other boundary representations in terms of assd , the metric less affected by the prediction thickness.

Original: Furthermore, it achieves competitive results for all other metrics, being the best performing method on Cityscapes and Mapillary Vistas in terms of F score.
Revised: Furthermore, it achieves competitive results for all other metrics, being the best performing method on Cityscapes and Mapillary Vistas in terms of F score.

Original: Throughout Table 2, it is possible to see that VT is the most stable method on the F score, being able to predict uniformly thin results.
Revised: Throughout Table 2, it is possible to see that VT is the most stable method on the F score, being able to predict uniformly thin results.

Original: On BSDS500, VT is the second best performing method on assd and F score, with a strong drop in overall performance given the dataset limitations.
Revised: On BSDS500, VT is the second best performing method on assd and F score, with a strong drop in overall performance given the dataset limitations.

Original: The VT ﬁeld, in particular, suffers from the inconsistent annotations as they change the morphology of the ﬁeld on and away from the boundaries, with errors that are not limited to the isolated pixel.
Revised: The VT ﬁeld, in particular, suffers from the inconsistent annotations as they change the morphology of the ﬁeld on and away from the boundaries, with errors that are not limited to the isolated pixel.

Original: Despite not achieving the best score on
Revised: Despite not achieving the best score on

Original: BSDS500, the result shows that VT is also able to predict low-level image contours without semantic meanings.
Revised: BSDS500, the result shows that VT is also able to predict low-level image contours without semantic meanings.

Original: To show its full capabilities in such task, it will be necessary to evaluate on a larger dataset with a clear deﬁnition of boundary, which is not currently available up to our knowledge.
Revised: To show its full capabilities in such task, it will be necessary to evaluate on a larger dataset with a clear deﬁnition of boundary, which is not currently available up to our knowledge.

Original: From the qualitative comparison between predictions in Fig.
Revised: From the qualitative comparison between predictions in Fig.

Original: 3, it is evident that the Vector Transform is the only method able to predict crisp boundaries without requiring any non-maximum suppression.
Revised: 3, it is evident that the Vector Transform is the only method able to predict crisp boundaries without requiring any non-maximum suppression.

Original: A particularly important evaluation here is that of DT versus VT.
Revised: A particularly important evaluation here is that of DT versus VT.

Original: We observe from the results that DT prediction in particular tends to detect thick boundaries, since a higher than 0 threshold is required to obtain enough recall, in turn leading to thick detection.
Revised: We observe from the results that DT prediction in particular tends to detect thick boundaries, since a higher than 0 threshold is required to obtain enough recall, in turn leading to thick detection.

Original: The discussions provided at the end of § is directly relevant to its performance.
Revised: The discussions provided at the end of § is directly relevant to its performance.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: Despite being an old computer vision problem, boundary detection remains an active research ﬁeld, both due to the many issues involved and its potential for other vision problems.
Revised: Despite being an old computer vision problem, boundary detection remains an active research ﬁeld, both due to the many issues involved and its potential for other vision problems.

Original: In this paper, we propose the Vector Transform and theoretically demonstrate its equivalence with a surface represen- tation of boundaries.
Revised: In this paper, we propose the Vector Transform and theoretically demonstrate its equivalence with a surface represen- tation of boundaries.

Original: The substantially different representation and corresponding approach of the
Revised: The substantially different representation and corresponding approach of the

Original: Vector Transform automatically solve two of the biggest challenges: class imbalance in the predic- tion and the ability to output crisp boundaries.
Revised: Vector Transform automatically solve two of the biggest challenges: class imbalance in the predic- tion and the ability to output crisp boundaries.

Original: We show the high performance capabilities of the representation and a few related tasks that can be tackled from such an approach, with extensive evaluations.
Revised: We show the high performance capabilities of the representation and a few related tasks that can be tackled from such an approach, with extensive evaluations.

Original: The formulation proposed in our work opens possibilities of new approaches on the use of boundaries and adds research questions on its applicability for several other related tasks.
Revised: The formulation proposed in our work opens possibilities of new approaches on the use of boundaries and adds research questions on its applicability for several other related tasks.

Original: A V ECTOR T RANSFORM IN P RACTICE
Revised: A V ECTOR T RANSFORM IN P RACTICE

Original: There are some aspects of Vector Transform which requires special attention in several non-ideal cases.
Revised: There are some aspects of Vector Transform which requires special attention in several non-ideal cases.

Original: The ﬁrst is the case of ground truth generation in datasets that were not designed with the interpretation of boundaries as surfaces.
Revised: The ﬁrst is the case of ground truth generation in datasets that were not designed with the interpretation of boundaries as surfaces.

Original: The second is the case of theoretical and practical aspects related to discontinuities and discretization.
Revised: The second is the case of theoretical and practical aspects related to discontinuities and discretization.

Original: G ROUND T RUTH G ENERATION
Revised: G ROUND T RUTH G ENERATION

Original: We now explain how in practice, we extract the Vector Transform from the ground truth boundary images to train the proposed method.
Revised: We now explain how in practice, we extract the Vector Transform from the ground truth boundary images to train the proposed method.

Original: For non-boundary pixels, the solution of the Vector Transform is clearly deﬁned as the direction towards the closest boundary.
Revised: For non-boundary pixels, the solution of the Vector Transform is clearly deﬁned as the direction towards the closest boundary.

Original: In practice, to mitigate for discretization-related errors, the closest boundary pixel is computed averaging the multiple closest ones.
Revised: In practice, to mitigate for discretization-related errors, the closest boundary pixel is computed averaging the multiple closest ones.

Original: This has the same effect of interpolating a planar surface on the set of closest boundary pixels and computing the normal direction to that surface.
Revised: This has the same effect of interpolating a planar surface on the set of closest boundary pixels and computing the normal direction to that surface.

Original: In the extreme case in which there are multiple equally distant boundaries, only one is chosen randomly as the closest one and the direction is computed towards it.
Revised: In the extreme case in which there are multiple equally distant boundaries, only one is chosen randomly as the closest one and the direction is computed towards it.

Original: For boundary pixels, there can be multiple scenarios to take into account depending on the dataset quality:
Revised: For boundary pixels, there can be multiple scenarios to take into account depending on the dataset quality:

Original: • If the boundary is taken from a segmentation image, it can be treated as a non boundary pixel with its direction computed toward the closest set of pixel outside its semantic class/instance object.
Revised: • If the boundary is taken from a segmentation image, it can be treated as a non boundary pixel with its direction computed toward the closest set of pixel outside its semantic class/instance object.

Original: • If the boundary is a manually annotated single pixel boundary, it is possible to consistently pair it to one of the neighboring non-boundary pixels and use its same vector direction.
Revised: • If the boundary is a manually annotated single pixel boundary, it is possible to consistently pair it to one of the neighboring non-boundary pixels and use its same vector direction.

Original: Any neighboring non-boundary pixel can be taken as long as the selection process remains consistent for each case.
Revised: Any neighboring non-boundary pixel can be taken as long as the selection process remains consistent for each case.

Original: • In case of thick boundaries, it is possible to devise a technique to assign a direction to each of its pixels - such as the direction of the closest non-boundary pixel and similarly to the previous point in undecided cases.
Revised: • In case of thick boundaries, it is possible to devise a technique to assign a direction to each of its pixels - such as the direction of the closest non-boundary pixel and similarly to the previous point in undecided cases.

Original: However, this is not a proper deﬁnition of a surface boundary and, in practice, it is not considered in any of the tested cases.
Revised: However, this is not a proper deﬁnition of a surface boundary and, in practice, it is not considered in any of the tested cases.

Original: D ISCONTINUITIES AND D ISCRETIZATION
Revised: D ISCONTINUITIES AND D ISCRETIZATION

Original: As explored in §3, the invertibility of the vector transform does not strictly satisfy at discontinuities.
Revised: As explored in §3, the invertibility of the vector transform does not strictly satisfy at discontinuities.

Original: Around the inﬁnitesimal neighborhood of the discontinuous boundary, the divergence is evaluated using an approximate normal.
Revised: Around the inﬁnitesimal neighborhood of the discontinuous boundary, the divergence is evaluated using an approximate normal.

Original: It can be observed that under a large number of circumstances, the transform can yield lower than -2 or -2 value in many discontinuities.
Revised: It can be observed that under a large number of circumstances, the transform can yield lower than -2 or -2 value in many discontinuities.

Original: However, the discretization and/or its combination with the discontinuities can impact the transform, where we may observe a higher than -2 divergence of the ﬁeld.
Revised: However, the discretization and/or its combination with the discontinuities can impact the transform, where we may observe a higher than -2 divergence of the ﬁeld.

Original: Fortunately, the representation provides a relatively large margin between the divergence of a non-boundary point from that of any other point.
Revised: Fortunately, the representation provides a relatively large margin between the divergence of a non-boundary point from that of any other point.

Original: Such a margin provides the robustness required in order to predict correct boundaries even in crowded regions.
Revised: Such a margin provides the robustness required in order to predict correct boundaries even in crowded regions.

Original: B A DDITIONAL R ESULTS A NALYSIS
Revised: B A DDITIONAL R ESULTS A NALYSIS

Original: In this section we report additional results with qualitative visualizations to support the observations in Section 5 and an additional experiment to analyze prediction proﬁles of DT versus VT measures around a predicted boundary.
Revised: In this section we report additional results with qualitative visualizations to support the observations in Section 5 and an additional experiment to analyze prediction proﬁles of DT versus VT measures around a predicted boundary.

Original: We show qualitative results obtained with every method on an image taken from each of the datasets.
Revised: We show qualitative results obtained with every method on an image taken from each of the datasets.

Original: From the qualitative comparison of predictions in Fig.
Revised: From the qualitative comparison of predictions in Fig.

Original: 4, it is evident that Vector Transform is the only method able to always predict crisp boundaries without requiring any non-maximum suppression.
Revised: 4, it is evident that Vector Transform is the only method able to always predict crisp boundaries without requiring any non-maximum suppression.

Original: This provides a signiﬁcant advantage, particularly in crowded regions where traditional methods require NMS post-processing to be used for downstream tasks.
Revised: This provides a signiﬁcant advantage, particularly in crowded regions where traditional methods require NMS post-processing to be used for downstream tasks.

Original: Therefore, our method provides a strong advantage when used at training time as an aide for different tasks.
Revised: Therefore, our method provides a strong advantage when used at training time as an aide for different tasks.

Original: Additionally, for the prediction proﬁle experiment we measure the divergence of VT prediction along the normal direction of the boundary at an increasing distance.
Revised: Additionally, for the prediction proﬁle experiment we measure the divergence of VT prediction along the normal direction of the boundary at an increasing distance.

Original: We compute the divergence versus distance for each predicted pixel.
Revised: We compute the divergence versus distance for each predicted pixel.

Original: For these measurements at each distance we compute the mean and standard deviation.
Revised: For these measurements at each distance we compute the mean and standard deviation.

Original: We perform the exact same experiment for the DT values instead of
Revised: We perform the exact same experiment for the DT values instead of

Original: VT divergence.
Revised: VT divergence.

Original: Both results are plotted in Fig.
Revised: Both results are plotted in Fig.

Original: The mean VT divergence (or DT value) versus the distance along the normal is plotted in black while the shaded region shows the standard deviation of the measure at each distance.
Revised: The mean VT divergence (or DT value) versus the distance along the normal is plotted in black while the shaded region shows the standard deviation of the measure at each distance.

Original: Note how VT divergence value quickly saturates to around 0 from
Revised: Note how VT divergence value quickly saturates to around 0 from

Original: -2 within a single pixel, while having an extremely low standard deviation.
Revised: -2 within a single pixel, while having an extremely low standard deviation.

Original: On the other hand, the
Revised: On the other hand, the

Original: DT prediction is mostly linear w.r.t distance by design but shows a large uncertainty around thedistance.
Revised: DT prediction is mostly linear w.r.t distance by design but shows a large uncertainty around thedistance.

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: P r ed i c t ed d i v e r gen c e
Revised: P r ed i c t ed d i v e r gen c e

Original: [Figure]
Revised: [Figure]

Original: C B OUNDARY D IRECTION E STIMATION
Revised: C B OUNDARY D IRECTION E STIMATION

Original: As our method outputs the VT for each pixel, when the boundaries are represented inside the image, it is natural to extract the boundary direction.
Revised: As our method outputs the VT for each pixel, when the boundaries are represented inside the image, it is natural to extract the boundary direction.

Original: Differently from other methods (Maninis et al., 2017), no additional module or post-processing is required to estimate the direction.
Revised: Differently from other methods (Maninis et al., 2017), no additional module or post-processing is required to estimate the direction.

Original: Furthermore, our method is able to predict continuous angles without the necessity to select from a discrete set of values.
Revised: Furthermore, our method is able to predict continuous angles without the necessity to select from a discrete set of values.

Original: In Fig.
Revised: In Fig.

Original: 6, we show a qualitative representation of our boundary direction estimation on three example images from Mapillary Vistas to have a visualization of the prediction quality.
Revised: 6, we show a qualitative representation of our boundary direction estimation on three example images from Mapillary Vistas to have a visualization of the prediction quality.

Original: From a quantitative standpoint, the predicting VT ﬁeld can achieve a root mean squared error on the estimated angle θ of 7 . 2 degrees.
Revised: From a quantitative standpoint, the predicting VT ﬁeld can achieve a root mean squared error on the estimated angle θ of 7 . 2 degrees.

Original: D S TRAIGHT L INE P ROPOSALS
Revised: D S TRAIGHT L INE P ROPOSALS

Original: Given that in our method each boundary pixel has a direction feature, it is natural to try and solve the task of straight line proposal generation for detection without any speciﬁc supervision.
Revised: Given that in our method each boundary pixel has a direction feature, it is natural to try and solve the task of straight line proposal generation for detection without any speciﬁc supervision.

Original: A straight line is considered a boundary for which the direction does not change in neighboring pixels.
Revised: A straight line is considered a boundary for which the direction does not change in neighboring pixels.

Original: To detect such points, it is possible to apply a simple algorithm:
Revised: To detect such points, it is possible to apply a simple algorithm:

Original: • First the VT ﬁeld is converted using only the absolute value of the two channels.
Revised: • First the VT ﬁeld is converted using only the absolute value of the two channels.

Original: In this way, differences of vectors having same orientation but opposite direction on the two sides of a boundary are removed.
Revised: In this way, differences of vectors having same orientation but opposite direction on the two sides of a boundary are removed.

Original: • Then the derivative of the two channels are approximated using a Sobel ﬁlter and their absolute values are summed in a pixelwise manner.
Revised: • Then the derivative of the two channels are approximated using a Sobel ﬁlter and their absolute values are summed in a pixelwise manner.

Original: In the obtained image, pixels with a high value indicate places where the orientation changes while the low values show constant orientation.
Revised: In the obtained image, pixels with a high value indicate places where the orientation changes while the low values show constant orientation.

Original: Therefore, we deﬁne a threshold ( t = 0 . 05 ) and consider part of a straight line the boundary pixels with a value below the threshold.
Revised: Therefore, we deﬁne a threshold ( t = 0 . 05 ) and consider part of a straight line the boundary pixels with a value below the threshold.

Original: In Fig.
Revised: In Fig.

Original: 7, we show some qualitative results obtained with the above method.
Revised: 7, we show some qualitative results obtained with the above method.

Original: It is possible to identify short straight lines detected in areas of the image without an apparent straight line.
Revised: It is possible to identify short straight lines detected in areas of the image without an apparent straight line.

Original: This is due to the small dimension ( 3 × 3) of the derivative kernel used to detect a straight line and could be solved by ﬁltering the prediction.
Revised: This is due to the small dimension ( 3 × 3) of the derivative kernel used to detect a straight line and could be solved by ﬁltering the prediction.

Original: We show the result without postprocessing as a proof that our method can be applied no matter what type of line detection is needed, from small segments to long lines. 
Revised: We show the result without postprocessing as a proof that our method can be applied no matter what type of line detection is needed, from small segments to long lines. [Figure]

Original: E S UPERPIXEL
Revised: E S UPERPIXEL

Original: The VT ﬁeld can also be used to create superpixels without any speciﬁc supervision.
Revised: The VT ﬁeld can also be used to create superpixels without any speciﬁc supervision.

Original: More specifically, when they are trained on semantically meaningful boundaries, they can be used to extract objects or object parts.
Revised: More specifically, when they are trained on semantically meaningful boundaries, they can be used to extract objects or object parts.

Original: This can be done without obtaining the partial result of boundaries, using only the ﬁelds and applying region growing algorithm on it.
Revised: This can be done without obtaining the partial result of boundaries, using only the ﬁelds and applying region growing algorithm on it.

Original: Speciﬁcally, we use the following algorithm:
Revised: Speciﬁcally, we use the following algorithm:

Original: • First, divergence is computed on the VT ﬁeld using a Sobel ﬁlter as done to obtain boundary pixels.
Revised: • First, divergence is computed on the VT ﬁeld using a Sobel ﬁlter as done to obtain boundary pixels.

Original: The high divergence values are the source points in the ﬁeld and are treated as centroids of the parts.
Revised: The high divergence values are the source points in the ﬁeld and are treated as centroids of the parts.

Original: In case there is a connected part with high divergence, it is considered to be a single centroid region.
Revised: In case there is a connected part with high divergence, it is considered to be a single centroid region.

Original: • Each pixel is associated to a two dimensional point based on its coordinates on the image grid.
Revised: • Each pixel is associated to a two dimensional point based on its coordinates on the image grid.

Original: The point is then moved (its coordinates are changed) following the opposite of the ﬁeld direction, towards the center and away from the border.
Revised: The point is then moved (its coordinates are changed) following the opposite of the ﬁeld direction, towards the center and away from the border.

Original: This continues until the algorithm converges or for a maximum number of steps.
Revised: This continues until the algorithm converges or for a maximum number of steps.

Original: This results in each pixel being associated with a point that has been moved to a different position from the original.
Revised: This results in each pixel being associated with a point that has been moved to a different position from the original.

Original: • Based on the position of the obtained point, the pixels are then assigned to the closest part centroid.
Revised: • Based on the position of the obtained point, the pixels are then assigned to the closest part centroid.

Original: Thanks to the complex shapes that the centroid regions can assume, the resulting clusters can have high structural complexity and represent complete objects or large parts.
Revised: Thanks to the complex shapes that the centroid regions can assume, the resulting clusters can have high structural complexity and represent complete objects or large parts.

Original: In the special case of pixels with an associated point that falls outside of the image border, an assignment to a part centroid cannot be done
Revised: In the special case of pixels with an associated point that falls outside of the image border, an assignment to a part centroid cannot be done.

Original: These pixels are then clustered using DBSCAN (Ester et al., 
Revised: These pixels are then clustered using DBSCAN (Ester et al., 1996)

Original: ) algorithm using as features the position of the associated points
Revised: algorithm using as features the position of the associated points.

Original: The technique beneﬁts from the possibility of being made highly parallel with the clustering algorithm that only needs to be applied on a limited number of well separated points.
Revised: The technique beneﬁts from the possibility of being made highly parallel with the clustering algorithm that only needs to be applied on a limited number of well separated points.

Original: We show some results obtained by grouping pixels based on the predicted Vector Transform in Fig.
Revised: We show some results obtained by grouping pixels based on the predicted Vector Transform in Fig.

Original: We compare to some well-know superpixel methods, i.e., Convolutional Oriented Boundaries (COB) (Maninis et al., 2017), Multiscale Combinatorial Grouping (MCG) (Arbel ´ aez et al., 2014), and Singlescale
Revised: We compare to some well-know superpixel methods, i.e., Convolutional Oriented Boundaries (COB) (Maninis et al., 2017), Multiscale Combinatorial Grouping (MCG) (Arbel´aez et al., 2014), and Singlescale

Original: Combinatorial Grouping (SCG) ( Arbel ´ aez et al., 2014), whose results are generated by thresholding the occlusion boundaries (Ultrametric Contour Map, UCM (Arbelaez et al., 2010)) using the optimal threshold of each method.
Revised: Combinatorial Grouping (SCG) (Arbel´aez et al., 2014), whose results are generated by thresholding the occlusion boundaries (Ultrametric Contour Map, UCM (Arbelaez et al., 2010)) using the optimal threshold of each method.

Original: It is visible that the VT ﬁeld can group an object with high complexity as a single superpixel, which can ease downstream tasks that beneﬁt from such a characteristic.
Revised: It is visible that the VT ﬁeld can group an object with high complexity as a single superpixel, which can ease downstream tasks that beneﬁt from such a characteristic.

Original: In contrast, previous methods tend to create a higher number of clusters whose boundaries are not always coincidental with the complete objects.
Revised: In contrast, previous methods tend to create a higher number of clusters whose boundaries are not always coincidental with the complete objects.

Original: This task further justiﬁes our re-deﬁnition of boundary detection.
Revised: This task further justiﬁes our re-deﬁnition of boundary detection.

Revised: F N ETWORK A RCHITECTURE AND T RAINING

Revised: In this section, we provide more details on the network architecture and the training procedures.

Revised: Figure 9 shows a schematic of the architecture used.

Revised: The architecture is chosen to take into account both high and low resolution details as commonly done by boundary detection methods (Liu et al., 2017; Xie & Tu, 2015).

Revised: More speciﬁcally, the use of a ResNeXt (Xie et al., 2017) block to process the high resolution signals and the upsampling stage are inspired by Deng et al.

Revised: Analyzing the architecture in details, we can identify four different sections:

Revised: • Downsampling phase : the image goes through two strided convolution layers which reduce the image resolution and increase the number of channels to 64 .

Revised: This is part of HRNet (Wang et al., 2020) but is represented separately for a better understanding.

Revised: • HRNet Main Body : this is the main part of HRNet (Wang et al., 2020) which extracts multiresolution features subsequently after the two strided convolutions.

Revised: The input and output of this block have the same resolution.

Revised: • ResNeXt Block : this is a single ResNeXt (Xie et al., 2017) block that is used to extract complex features to be used while retrieving the full resolution.

Revised: More speciﬁcally, the block has cardinality 4, using the same notation as Xie et al.

Revised: • Upsampling phase : the ﬁnal part of the network takes as input, the output of HRNet and upsamples it to formulate full resolution predictions.

Revised: First, the input is bilinearly upsampled and the result is merged to the output of the ResNeXt block concatenating the two and applying a pixel-wise fully-connected layer.

Revised: Finally, the result is further bilinearly upsampled and one ﬁnal

Revised: convolution layer is used to predict the output representation.

Revised: The training protocol used for the network resembles the one used in Panoptic-DeepLab (Cheng et al., 2020).

Revised: More speciﬁcally, we apply an initial learning rate of 0 .

Revised: 001 , the ’poly’ learning rate scheduler (Liu et al., 2015) and a batch size of 32 .

Revised: The images are augmented using random resizing, random horizontal ﬂipping and randomly cropping the resulting image to the size of 512 × 512 irrespective of the dataset.

Revised: The dimensions for randomly resizing the shortest side are from the set of dimensions { 512 , 640 , 704 , 832 , 896 , 1024 , 1152 , 1216 , 1344 , 1408 , 1536 , 1664 , 1728 , 1856 , 1920 , 2048 } .

Revised: Inference, instead, is done at a single image resolution irrespective of the method applied.

Revised: For

Revised: Cityscapes (Cordts et al., 2016) and Mapillary Vistas (Neuhold et al., 2017) the shortest edge has dimension 1024 pixels and for Synthia (Ros et al., 2016)

Revised: it is 960 pixels.