<figure><img src="image_1.png" /></figure>

HAL
open science

# Object Detection with Component-Graphs in Multi-band Images: Application to Source Detection in Astronomical Images

Thanh Xuan Nguyen, Giovanni Chierchia, O Razim, R Peletier, Laurent Najman, Hugues Talbot, Benjamin Perret

► To cite this version:

Thanh Xuan Nguyen, Giovanni Chierchia, O Razim, R Peletier, Laurent Najman, et al., Object Detection with Component-Graphs in Multi-band Images: Application to Source Detection in Astronomical Images. IEEE Access, 2021, 9, pp.156482-156491. 10.1109/ACCESS.2021.3128519. [hal-03429058v2](https://hal.science/hal-03429058v2)

HAL Id: hal-03429058

https://hal.science/hal-03429058v2

Submitted on 19 Nov 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.

Digital Object Identifier 10.1109/ACCESS.2017.DOI

# Object Detection with Component-Graphs in Multi-band Images: Application to Source Detection in Astronomical Images

T. X. NGUYEN<fnref n="1" />, G. CHIERCHIA<fnref n="1" />, O. RAZIM<fnref n="2" />, R. F. PELETIER<fnref n="3" />, L. NAJMAN<fnref n="1" /> (Senior Member, IEEE), H. TALBOT<fnref n="4" /> (Member, IEEE), AND B. PERRET<fnref n="1" />

Corresponding author: L. Najman (e-mail: laurent.najman@esiee.fr).

This research was funded by the European Union Horizon 2020 program under the Marie Skłodowska-Curie grant agreement No. 721463 to the SUNDIAL ITN network and the Programme d'Investissements d'Avenir (LabEx BEZOUT ANR-10-LABX-58).

**ABSTRACT** In the context of mathematical morphology, component-graphs are complex but powerful structures for multi-band image modeling, processing, and analysis. In this work, we propose a novel multi-band object detection method relying on the component-graphs and statistical hypothesis tests. Our analysis shows that component-graphs are better at capturing image structures compared to the classical component-trees, with significantly higher detection capacity. Besides, we introduce two filtering algorithms to identify duplicated and partial nodes in the component-graphs. The proposed method, applied to the detection of sources on astronomical images, demonstrates a significant improvement in detecting faint objects on both multi-band simulated and real astronomical images compared to the state of the art.

**INDEX TERMS** Morphology, Component-graphs, Object detection, Astronomical Object

## I. INTRODUCTION

In mathematical morphology, component-trees (CTs) and component-graphs (CGs) are classical structures for image modeling and analysis. These structures model images as hierarchical representations using successive thresholding where each node is a connected component. All CT variants (Min-Tree, Max-Tree [1], [2], Tree of Shape [3]) benefit from efficient construction and filtering algorithms [4], [5]. They have diverse applications related to connected filtering, object detection, and segmentation, but those are limited to single-band image processing. Extension to multi-band image processing usually requires a total vectorial order (such as lexicographic ordering, reduced ordering) that is usually ad-hoc and application-dependent [6], [7].

On the other hand, CGs are designed to handle multi-band images by relying on partial orderings [8], [9]. Beyond the classical multivariate extensions of CTs, CGs efficiently capture the whole structural information of multi-band images as directed acyclic graph (DAG) variants. Such DAG variants are more general and more powerful at the cost of a higher construction complexity. Component-graphs have been increasingly considered for detection and segmentation applications in the field [10]. This work explores the use of the component-graphs for multi-band object detection and proposes an application to astronomical images.

<figure><img src="image_3.png" /><figcaption>FIGURE 1: Component Graph Objects (CGO) filtering method using component-graphs.</figcaption></figure>

In astronomy, object/source extraction is fundamental preliminary stage before entering any further analysis. The chal-

<footnote n="1">¹LIGM, Univ Gustave Eiffel, CNRS, ESIEE Paris, F-77454 Marne-la-Vallée, France</footnote>

<footnote n="2">²Department of Physics, University of Naples Federico II, Napoli, Italy</footnote>

<footnote n="3">³Kapteyn Astronomical Institute, University of Groningen, Groningen, the Netherlands</footnote>

<footnote n="4">⁴Université Paris-Saclay, CentraleSupélec, Inria, France</footnote>
<figure><img src="image_4.png" /><figcaption>FIGURE 2: Object deblending capacity: A two-band image containing three overlapping circles. The middle circle appears in the CG as an isolated node while it is merged with adjacent regions in the CT of separate bands. This color information in the CG can help deblend overlapping objects.</figcaption></figure>

lenge is to develop efficient and automated tool for the large datasets/surveys. The most often used source finder is SExtractor [11], an efficient and easy to use application. However, it fails at detecting some faint and diffuse objects. For this reason, MObject/Sourcerer [12], [13] was introduced to improve the SExtractor thresholding strategy by using a CT structure. More precisely, MObject/Sourcerer relies on statistical tests to identify nodes of a Max-Tree that are significantly different from the background. MObject/Sourcerer has already shown its capability at detecting faint astronomical sources [14] while requiring far less parameter tuning than SExtractor. However, both methods focus on single-band processing while most optical astronomical surveys are multi-band. To handle such images, that are expected to lead to an increased sensitivity, we propose to generalize the detection method based on statistical testing to CGs. The main challenge is to effectively leverage multi-bands to filter relevant information from the richer component-graph structure.

This article extends our previous work [15] on Component Graph Objects (CGO) - a novel multi-band object detection framework with comprehensive analyses on both simulated and real datasets. The overview of the proposed framework is illustrated in Fig. 1. Intuitively, CGO models multi-band images as component-graphs, then filters relevant nodes on the component-graph, and transforms extracted information back into the knowledge space.

By utilizing the CG structure, CGO attempts to improve object detection sensitivity and to improve object deblending capacity. First, the use of multi-band information improves detection of lower signal-to-noise objects at the same level of confidence. Second, the richness of the CG helps to deblend overlapping objects that would have been merged with single band analyses (see Fig. 2). In contrast to these advantages, the component-graph is no longer a tree (see Fig. 4), but a directed acyclic graph (DAG), which is significantly more challenging to process than the classical component-trees [10].

After some definitions in Sec. II, we introduce our new method, called CGO in Sec. III, which is based on a set of multi-band node attributes and two algorithms for duplicated node differentiation and partial node detection. Sec. IV proposes an application of CGO to detect sources on astro-

nomical images. Experimental results in Sec. V show that CGO can detect faint sources on both simulated and real multi-band images, with significantly better precision and recall than the state-of-the-art method [12], [14]. In addition, Sec. V also demonstrates that the component-graph is better at capturing image structure comparing to the component-tree.

## II. COMPONENT-GRAPHS

We recall some definitions on graphs and component-graphs. In-depth presentations of the component-graph can be found in [9], [10].

### A. ORDER RELATIONS

Order relation is essential to define the relationships between components in the morphological structures. Given a finite set of elements $\Gamma$, a binary relation $\leq$ on $\Gamma$ is an order relation and $(\Gamma, \leq)$ is a finite ordered set if $\leq$ is reflexive, transitive, and anti-symmetric. We say that $\leq$ is a partial order relation, and that $(\Gamma, \leq)$ is a partially ordered set, if there exist non-comparable elements in $(\Gamma, \leq)$, i.e., $\exists x, y \in \Gamma, (x \not\leq y \land y \not\leq x)$. The order relation $\leq$ is a total order relation, and $(\Gamma, \leq)$ is a totally ordered set, if $\forall x, y \in \Gamma, (x \leq y \lor y \leq x)$.

### B. GRAPHS AND IMAGES

A graph $G$ is a pair $(V, E)$, where $V$ is a finite set and $E$ is a set of pairs of distinct elements of $V$, i.e., $E \subseteq \{(x, y) \subseteq V \mid x \neq y\}$. An element of $V$ is called a vertex of $G$, an element of $E$ is called an edge of $G$.

Given a graph $G = (V, E)$ we say that a sequence of elements $(x_0, \dots, x_n) \in V$ is a path in $V$ from $x_0$ to $x_n$ if $\{x_{i-1}, x_i\} \in E, \forall i \in \{1, \dots, n\}$. A subset $V' \subseteq V$ is said to be connected if for any two distinct elements $x, y \in V'$, there exists a path from $x$ to $y$. A connected component of $G$ is a maximal connected subset of $V$. The set of all connected components of $G$, denoted as $C[G]$, is a partition of $V$.

Let $F$ be a function from $V$ to a nonempty set $\mathbb{V}$ equipped with an order relation $\leq$. We say that $(G, F)$ is a vertex-valued graph (or valued graph).

In practice, given a valued graph $(G, F)$, the graph $G$ can be used to represent the domain of an image where each vertex corresponds to a pixel and where edges correspond to the adjacency relation between pixels [16]. The function $F$ then represents an image associating a possibly multivariate value to any pixel/vertex.

### C. COMPONENT-GRAPHS

Given a valued graph $(G, F)$, we define the threshold set

$$V_v = \{x \in V \mid F(x) \geq v\}, \qquad (1)$$

where $v \in \mathbb{V}$. The threshold set $V_v$ induces a subset $E_v = \{(x, y) \in E \mid x, y \in V_v\}$ and a sub-graph $G_v = (V_v, E_v)$. The set of connected components of the sub-graphs $G_v$ of $G$ for all $v \in \mathbb{V}$ is denoted as

$$\Psi = \bigcup_{v \in \mathbb{V}} C[G_v]. \qquad (2)$$
<figure><img src="image_5.png" /><figcaption>FIGURE 3: Component-tree example: (a) A grayscale image with values in $V = \{0, 1, 2, 3\}$; (b) The Max-tree of the image; and (c-f) The threshold sets $V_v$ for $v \in V$. The letters (R, A, B, D) refer to the connected components corresponding to the nodes in the tree. Note that the connected components in figures (c-f) are down-scaled by a factor of two for visualization purpose.</figcaption></figure>

* If $(V, \leq)$ is totally ordered, the partially ordered set $(\Psi, \subseteq)$ forms a Max-Tree of the valued graph $(G, F)$ (see Fig. 3).

* If $(V, \leq)$ is partially ordered, the partially ordered set $(\Psi, \subseteq)$ forms a component-graph, denoted by $\Theta$, of the valued graph $(G, F)$ [8] (see Fig. 4).

In our work, we use a simplified version of the CG, denoted $\ddot{\Theta}$ (see Fig. 4c), where its set of connected components

$$ \ddot{\Psi} = \left\{ X \in \Psi \mid \bigcup_{\substack{Y \in \Psi \\ Y \subseteq X}} Y \neq X \right\} \quad (3) $$

contains only the connected components that contribute to the visibility of the image $F$ [9]. The CG $\Theta$ and the CG $\ddot{\Theta}$ are both directed acyclic graphs. The set $\ddot{\Psi}$ is a subset of the set $\Psi$. The CG $\Theta$ associated to the set $\Psi$ containing all valued connected components in the image is the most informative structure, but also the most expensive to construct $(O(n^3))$. Since the CG $\ddot{\Theta}$ takes into account only visible components from the image, it is less expensive to construct $(O(n^2))$ than the full CG $\Theta$ [10]. In the remainder, we always use the simplified CG $\ddot{\Theta}$. An element of $\ddot{\Psi}$ is called a node of the CG $\ddot{\Theta}$. Note that all three of CT, $\Theta$ and $\ddot{\Theta}$ are lossless representations of the same image and so no information is lost in spite of the simplification.

### D. COMPONENT-GRAPH CONSTRUCTION

For the sake of completeness, we briefly revisit the component-graphs construction algorithms following [9], [10]. Since the component-graph relies on partial ordering, it raises structural and algorithmic differences compared to the classical component-trees. Generally, all component-graph

variants can be built with greedy strategy, in which propagations are performed from all possible connected components. In this paper, we use an alternative approach to construct the component-graph $\ddot{\Theta}$ more efficiently. In this component-graph, all components are visible, i.e., each component contains at least one pixel that is not covered by any smaller component. From this observation, our alternative algorithm builds $\ddot{\Theta}$ by finding the right node for each pixel. First, it models the input image as a region-adjacency graph (RAG) where each vertex is a flat-zone and edges represent flat-zone neighbors. The RAG vertices are kept in a priority queue which guarantees that children nodes are visited before parent nodes. Then, the nodes in the queue are visited to build $\ddot{\Theta}$ from leaves (pixel levels) to root.

### E. COMPONENT-GRAPH ATTRIBUTES

Given the component-graph $\ddot{\Theta}$ of the valued graph $(G, F)$, node attributes are essential for node filtering algorithms. Let $N$ be a node of the CG $\ddot{\Theta}$, we formalize the basic attributes of the CG as follows:

* The level $L(N)$ is the infimum of vertex values in the node $N$:

$$ L(N) = \bigwedge \{F(x), x \in N\}. \quad (4) $$

* The area $a(N)$ is the number of vertexes belonging to that node:

$$ a(N) = |N|, \text{ i.e., the cardinality of } N. \quad (5) $$

* The parents $\text{parents}(N)$ are the smallest nodes of the CG $\ddot{\Theta}$ larger than the node $N$:

$$ \text{parents}(N) = \min\{X \in \Psi \mid N \subseteq X\}. \quad (6) $$

As a consequence of the partial order relation $\leq$, a node $N$ may have several parent nodes.

* The significance $\text{sn}(N, b)$, $\text{sn}_{\text{syn}}(N)$ and $\text{sn}(N)$ are predicates saying whether the node $N$ is significant respectively in the $b$-th band, in the synthesized band, and in all bands, their designs are upon applications. For instance, a measure of eccentricity can be used for elongated object filtering, or compactness can be used for round object detection. Our significance definitions targeting astronomical sources are introduced in Section IV-A.

* The closest significant ancestors $\text{sn}_{\text{anc}}(N)$ are the smallest significant ancestors of $N$:

$$ \text{sn}_{\text{anc}}(N) = \min\{X \in \ddot{\Theta} \mid N \subseteq X \text{ and } \text{sn}(Y)\}. \quad (7) $$

Because of the partial order $\leq$, a node $N \in \ddot{\Theta}$ may have several closest significant ancestors.

### III. FILTERING THE COMPONENT-GRAPH

We introduce Component-Graph Objects (CGO), a method to handle multi-band object detection with CG. We first address the transition of object detection from CT to CG, then we present the filtering algorithm.
<figure><img src="image_6.png" /><figcaption>FIGURE 4: Component-graphs example: (a) A two-band image with multivariate values in $V = \{(0,0), (2,1), (1,2), (3,3)\}$ equipped with the marginal partial order relation $\le_m$; (b-c) The CG $\Theta$ and the simplified CG $\tilde{\Theta}$ of the image; and (d-h) The threshold sets $V_v$ for $v \in V$. The CG $\tilde{\Theta}$ does not contain the node A because A is invisible (behind B and C) in the input image.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>FIGURE 5: CT and CG structure differences: (top) A two-band input image containing a single faint source and its ground-truth; (bottom) The CG (of the both bands) and the CT (of the first band) of the input image where the node color represents the similarity between the ground-truth and the node. Note that the parent relations are not drawn, to simplify the illustrations.</figcaption></figure>

The CG is a directed acyclic graph while the CT forms a tree, as the name implies. Fig. 5 visualizes the structural differences between CT and CG via their similarity maps between each node and the ground-truth node of a single source image. The similarity is measured by the Intersection over Union (IoU) metric, defined as the area of the intersection divided by the area of the union of the two components. In both cases, there exist many candidate nodes (with high IoU score) associated with the single object in the input image. For the CT, filtering objects from those similar nodes is straightforward, as good candidate nodes of an object form a branch in the tree. On the other hand, the DAG structure of the CG allows the candidate nodes to form many branches associated with a single object.

We now present a novel algorithm to deal with the multi-band object detection with the CG. The main filtering algorithm Alg. 1 takes three inputs: the component-graph $\tilde{\Theta}$ representing the input image; the significance attribute $sn()$ identifying significant nodes; and the function $differ()$ measuring the dissimilarity between nodes. It outputs a list of object nodes. The algorithm is composed of two filtering steps which are described in detail in the two following sections. Intuitively, the first filtering attempts to remove duplicated nodes in the component-graphs. The aim of the second step is then to remove partial nodes referring to the same object.

## A. DUPLICATED OBJECT DETECTION

In the morphological data structures (the CT and the CG), objects appear differently at different thresholding levels as sequences of significant nodes. For instance, in the case of a single-band input, the object in Fig. 3a is represented by

### Algorithm 1 Filtering the component-graph $\tilde{\Theta}$

Require: Component-Graph $\tilde{\Theta}$.

Require: Function $sn()$ determines significant node.

Require: Function $differ()$ distinguishes two nodes.

Ensure: List of object nodes.

/* Filter duplicated nodes */

1: **for** $N \in \{X \in \tilde{\Theta} | sn(X)\}$ from root to leave **do**

2: &nbsp;&nbsp;&nbsp;&nbsp;**if** $sn_{anc}(N) = \emptyset$ or differ(N, Y) $\forall Y \in sn_{anc}(N)$ **then**

3: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**objs** $\leftarrow$ **objs** $\cup$ **N**

/* Filter partial nodes */

4: **for** node $N \in \text{objs}$ **do**

5: &nbsp;&nbsp;&nbsp;&nbsp;**if** $sn_{syn}(N)$ **then**

6: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**if** $partial(N, \tilde{\Theta}, b) \forall \text{band b such that } sn(N, b)$ **then**

7: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**objs** $\leftarrow$ **objs** $\setminus \{N\}$

8: **return** **objs**

the three nodes $\{A, B, D\}$ in the Max-Tree (see Fig. 3b). In the case of a multi-band input (see Fig. 6a), three nodes $\{B, C, D\}$ in the component-graph $\tilde{\Theta}$ (see Fig. 6b) may correspond to one or two objects. Specifying objects among those potentially overlapping nodes is not straightforward on either the CT or the CG.

In the context of the CT with a total order, the function $differ()$ can rely on the main branch assumption [12]: a node and its main branch node reside in the same object, where the main branch node is defined as the largest significant
<figure><img src="image_8.png" /><figcaption>FIGURE 6: Duplicated objects detection: (a) A two-band image, (b) The CG $\ddot{\Theta}$, where significant nodes are marked yellow. Branch 1 and Branch 2 are incomparable and growing to the same leaf node.</figcaption></figure>

descendant of a node. A sequence of main branch nodes following a node forms the main branch. Then, all nodes in a main branch represent the same object. Back to the single-band example (in Fig. 3a and Fig. 3b)), three nodes {$A, B, D$} simply belong to the main branch ($A \to B \to D$) in the Max-Tree, then they all represent a single object.

However, in the context of the CG with partial orders, there may exist several branches containing non-comparable nodes belonging to a single object. The main branch assumption is thus not enough to differentiate these branches in the new multi-band context. For example in Fig. 6a and Fig. 6b, both branches ($B \to D$) and ($C \to D$) in the CG $\ddot{\Theta}$ may correspond to one or two objects. We propose to generalize the main branch approach by using a generic function that measure dissimilarity between two nodes and that should be designed specifically for each application. Alg. (1) then identifies candidate nodes by browsing significant nodes from the root to the leaves of the CG (line 1): If the current significant node does not have any significant ancestor or if it is significantly different (according to the function differ()) from all its significant ancestors then it is an object candidate (line 2); Otherwise, the node is considered a duplicated node. A practical differ() function for astronomical images is described in IV-B.

## B. PARTIAL NODE DETECTION

In the CG, significant adjacent nodes can be non-comparable when marginal orders in separate bands disagree. Those nodes can be captured as isolated objects whereas they may belong to the same object. An example is shown in Fig. 7, where three significant adjacent nodes E, F, G are non-comparable in a two-band image, but they appear to be detected as three separated objects associated to the three branches (R, E), (R, F), (R, G).

We propose a partial detection step to validate the signifi-

<figure><img src="image_9.png" /><figcaption>FIGURE 7: Partial object detection: (a) A two-band image I valued on $V = \{(0,0), (2,0), (0,2), (1,1)\}$ equipped with the marginal order relation $\le_m$; (b) The CG $\ddot{\Theta}$ of the input image where yellow nodes are significant; and (c-f) The threshold sets $V_v$ for $v \in V$. Considering the first band, nodes E and G should be considered as two parts of a single object, but they are isolated because of the order disagreement in the two-band space. The situation is similar for nodes F and G in the second band.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>FIGURE 8: The max-tree of N, adjacent N', and the union $U = N \cup N'$ in band b.</figcaption></figure>

cance of the candidates band-by-band to eliminate the partial nodes. The algorithm Alg. (1) checks each candidate node N (line 4): If N is significant in the synthesized band, then N is an object node (line 5); Otherwise, the node N is partial if N is partial in all the bands where N is significant (line 6-7) (Alg. (2) determines whether a node is partial in a specific band b).

The idea of Alg. (2) is to test whether N expands any adjacent node of N. For each node N' adjacent to N (line 1), we look at the Max-Tree of N, N' and the union $U = N \cup N'$, see Fig. 8:

* If $L(N)_b > L(N')_b$, N is an isolated significant node in the Max-Tree in band b, then N is an object node regardless of the significance of the union U (line 2-3), i.e., N is not partial, see Fig. 9(a-b).

* If $L(N)_b \le L(N')_b$, then N is included as a part of the union U in the Max-Tree in the band b, see Fig. 9(c-f), there are two possibilities that make the candidate node N be partial: First, the union U is not significant in band
<figure><img src="image_11.png" /><figcaption>FIGURE 9: The possible links between nodes in the max-tree of N, N' adjacent to N, and U = N ∪ N' in the band b: (a-b) L(N)_b > L(N')_b and (c-f) L(N)_b ≤ L(N')_b.</figcaption></figure>

b, then N becomes part of the non-significant union U (line 4-5), see Fig. 9(c-d); Second, the union U and N' are both significant, then N is part of the object node U which is represented by N' (line 6-7), see Fig. 9(e); Otherwise, N' is non-significant while the union U is significant, then N remains as object node, i.e., N is not partial, see Fig. 9(f).

Back to the example in Fig. 7 where *E*, *F*, *G* are the three candidate nodes in the graph $\ddot{\Theta}$. The partial detection would validate *E* as isolated object because it is significant in the first band and it merges to the root in the second band. Similarly, *F* is also marked as isolated object. For *G* in the first band, it expands into the union *C* = *G* ∪ *E*: If the union *C* is significant, then *G* is part of the significant union *C*; Otherwise, *G* is non-object node. The situation is similar for *G* and *F* in the second band. All in all, *E* and *F* are kept as object nodes.

Practically, the adjacent nodes Adj(N) of the node N is costly to retrieve from the CG (with complexity $O(n^2)$ with n the number of nodes). In this work, we approximate Adj(N) by the adjacent sibling set, which is more efficient to retrieve. In the component-graph, the sibling set of a node is reachable in constant time. Theoretically, checking whether two nodes are adjacent costs $O(n)$. Then the complexities of Alg. 1 and Alg. (2) are respectively $O(n.k.m)$ and $O(n.k)$ with n the number of nodes in the component-graph, k the average number of siblings of the significant nodes, and m the number of candidate nodes. In general, the number m and k are significantly smaller than the number of nodes n.

## IV. APPLICATION TO ASTRONOMICAL IMAGES

We describe an application of the proposed method CGO to detect sources on multi-band astronomical images. As the CGO filtering algorithm requires, we design a significant attribute (IV-A) and a node dissimilarity measure (IV-B).

### A. SIGNIFICANCE ATTRIBUTE OF ASTRONOMICAL SOURCES

For astronomical images, we extend the idea of the MObject significance test [12] to the multi-band context. This significance measure is based on a chi-square distribution of the brightness of the component pixels, assuming additive white Gaussian noise. More precisely, the area of the component

**Algorithm 2** Partial detection: partial($\ddot{\Theta}$, N, b)

**Require:** $\ddot{\Theta}$, a component-graph.

**Require:** $N \in \ddot{\Theta}$, a candidate node.

**Require:** *b*, a significant band.

**Ensure:** true if N is a partial node in band *b*.

1. **for** $N' \in Adj(N)$ **do**

2. **if** $L(N)_b > L(N')_b$ **then**

3. **continue**

4. **if** not $sn(N \cup N', b)$ **then**

5. **return true**

6. **if** $sn(N \cup N', b)$ and $sn(N', b)$ **then**

7. **return true**

8. **return false**

(i.e., the number of pixels) is the number of degrees of freedom of the chi-square distribution where each pixel brightness is considered (a contrario) as an independent normal random variable. Its computation relies on two component-attributes: the node normalized power and the node area. Let N be a node. The **node power** is the sum of the squared difference between the node pixel values and the level of the parents. Since a node in the CG $\ddot{\Theta}$ may have several parents, this definition uses the supremum (average, infimum, max area node can also be used) of the parent levels as a reference:

$$ \mathcal{E}(N) = \sum_{x \in N} \left( \mathbf{F}(x) - \bigvee_{y \in \text{parents}(N)} \mathbf{L}(y) \right)^{\circ 2}, \quad (8) $$

where $\bigvee$ is the supremum operator and $\circ$ is the element-wise power.

The **node normalized power** normalizes the node power by the local background variance:

$$ \mathcal{E}'(N) = \mathcal{E}(N) \oslash \hat{\sigma}_{bg}^2, \quad (9) $$

where $\oslash$ is element-wise division; $\hat{\mu}_{bg}$, $\hat{\sigma}_{bg} \in \mathbb{R}^c$ stand for the mean and the standard deviation of the background of the image **F**. The background is approximated by the combination of flat tiles which are determined using D'Agostino's $K^2$ test [12].

The **node significance** relies on hypothesis testing. Let *b* be a band number, the single band significance test $sn(N, b)$ is the same as in [12]:

$$ sn(N, b) = \mathcal{E}'(N)_b > \text{cdf}\chi^2(\alpha, a(N)), \quad (10) $$

where $\mathcal{E}'(N)_b$ is the normalized node power in band *b*, cdf$\chi^2()$ is the chi-square cumulative distribution function, $\alpha$ is a significance level, and $a(N)$ is the area of the node N. The test is extended to a multi-band significance test $sn(N)$ defined as follows:

$$ sn(N) = (\exists b \in [0, c), sn(N, b)) \text{ or } (sn_{\text{syn}}(N)), \quad (11) $$

where

$$ sn_{\text{syn}}(N) = \left( \sum_{b=0}^{c} \mathcal{E}'(N)_b > \text{cdf}\chi^2(\alpha, c a(N)) \right), \quad (12) $$
is the synthesized band significance test and c the number of band.

A node is considered significant if it is so in a single band or in the synthesized band. We leverage the multi-band information in the synthesized band to detect significant nodes even if their signal in separated bands are all non-significant. For checking the separated bands, the first term in Eq. (11) guarantees to capture whatever the single-band significance test can capture in each band. For checking the synthesized band, the second term in Eq. (11) takes into account the combined power attribute to determine whether the combined signal is statistically significant. Since the multi-band test evaluates all bands simultaneously, it can indeed detect cases where a node is non-significant in all separated bands, but is significant in the synthesized band.

## B. DUPLICATED ASTRONOMICAL SOURCE DETECTION

The center of astronomical sources is usually brighter and better localized than the outer parts. This observation means that, in the CG, two significant nodes with similar centers likely represent the same object. We define a predicate differ() expressing whether two nodes belong to the same object as

$$ \text{differ}(N_1, N_2) = \| \text{center}(N_1) - \text{center}(N_2) \| < r, \quad (13) $$

where $N_1, N_2 \in \tilde{\Theta}$ are two nodes, the function center() returns the center pixel of a node, e.g. the brightest pixel of the node, and r is a thresholding radius. The center pixel could also be defined as the center of mass or as the center of the best fitting ellipse of the node.

## V. EXPERIMENTS

This section shows the relevance of our proposed method for object detection in astronomical images. In all the experiments, the graph G is the classic 4-connected one. We compare CGO with the state-of-the-art method MObject [14] on simulated and real images in the following experiments:

* **Statistical Test Boundaries** (sec. V-A) investigates the detection boundary difference between single-band and multi-band statistical tests.

* **Detection Capacity** (sec. V-B) studies how well the component structures are preserved in the CT and the CG via simulations.

* **Evaluation on a simulation** (sec. V-C) assesses the methods on simulated astronomical images.

* **Evaluation on real images** (sec. V-D) assesses the methods on real astronomical surveys.

### A. STATISTICAL TEST BOUNDARIES

Both CGO and MObject rely on statistical hypothesis testing to identify significant components. It is important to formalize and visualize the difference between the tests: the single-band test (with respect to MObject) and the multi-band test (with respect to CGO). If we assume that the noise is Gaussian, then the node normalized power attribute follows

<figure><img src="image_12.png" /><figcaption>FIGURE 10: Visualization of statistical test rejection boundaries of single-band and multi-band components at the same level of significance $\alpha = 10^{-6}$.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>FIGURE 11: Detection upper bounds of the morphological structures.</figcaption></figure>

a chi-square distribution. At a given significant level $\alpha$, the rejection boundary for the statistical test is then equal to

$$ b(n) = \{ (a, p) \in \mathbb{R}^2 | a \in \mathbb{N}, 1 - \text{cdf}\chi^2(n \times a, n \times p) = \alpha \}, \quad (14) $$

where n is number of bands; a, p denote the node area and the normalized power; and cdf$\chi^2()$ is the chi-square cumulative distribution. Fig. 10 and Eq. (14) distinctly reveal the theoretical gap between rejection boundaries, i.e., at the same confidence level, the multi-band statistical test is more sensitive to weak signal than the single-band statistical test. We note that the multi-band gain is not linear with the number of bands.

### B. UPPER BOUND DETECTION CAPACITY OF THE CT AND THE CG

To detect target objects, it is critical that the morphological representation of the image capture them as nodes. In this experiment, we assess how well objects are captured in the CT and the CG by studying their node similarity upper
<figure><img src="image_14.png" /><figcaption>FIGURE 12: Single Source Simulation: (left) A synthetic image with signal-to-noise ratio −0.93 and (right) The corresponding ground truth.</figcaption></figure>

bounds on a synthetic dataset. For a set of nodes $\ddot{\Psi}$ and a ground-truth node $gt$, we define the similarity upper bound as

$$ \mathrm{Sup}(\ddot{\Psi}, gt) = \max_{N \in \ddot{\Psi}} J(N, gt), \quad (15) $$

where $J$ stands for the Jaccard distance between two components. As we can see, the higher the similarity upper bound, the more likely target objects can be detected and segmented properly. The node associated to the similarity upper bound $\mathrm{Sup}(\ddot{\Psi}, gt)$ can be interpreted as the best object-like node existing in the set $\ddot{\Psi}$.

We analyze the similarity upper bounds on a single source simulation, as shown in Fig. 12. The simulation includes 10⁴ three-band images of size (50, 50) pixels. Each image contains a single point source with Gaussian noise, and the ground-truth is defined as the region covering 99% of the source brightness. The component-trees of the separate bands, the average band, and the component-graph of the three-band image are constructed. Fig. 11 shows the average similarity upper bounds with respect to the signal-to-noise ratio of the simulated sources. On this synthetic dataset, the component-graph provides higher similarity upper bounds than the component-tree, i.e., it has better detection capacity comparing to the component-tree.

### C. EVALUATION ON AN ASTRONOMICAL SIMULATION

This experiment assesses detection capacity of the proposed CGO and the state-of-the-art MTObject [12] [14] for astronomical object detection on a multi-band simulated dataset. For fair comparisons, we suggest M-MTObject in Sec. V-C3 - a straightforward extension of the default single-band MTObject to process multi-band images.

#### 1) FDS Simulation

We rely on a simulated three-band astronomical dataset with ground-truth imitating the Fornax Deep Survey [17], a wide field imaging survey of the Fornax Cluster using ESO's VST telescope. It contains 1500 stars and 4000 galaxies. Because the $\ddot{\Theta}$ construction is computationally expensive ($O(n^2)$), we performed the simulation into tiles of size (500, 500) pixels with an overlapping of 250 pixels. For each tile, we have a

<figure><img src="image_15.png" /><figcaption>FIGURE 13: FDS simulation: (left) The three-band simulated image and (right) the ground-truth map represent stars/galaxies as separate color blocks.</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>FIGURE 14: Evaluation on the FDS Simulation.</figcaption></figure>

three-band image with a ground-truth segmentation. The full-size simulation is visualized in Fig. 13.

#### 2) Metric

We use precision, recall, and F1-score, as in [14]. The evaluation matches at most one detected object in the detection map to each target object in the ground-truth map. Each target object in the ground-truth map is represented by its brightest pixel called its representative pixel, hence each representative pixel is included in at most one object in the detection map. If a detected object contains several representative pixels of different target objects, then the detected object is associated to the target object with the brightest representative pixel.

#### 3) M-MTObject

Along with the state-of-the-art single-band MTObject [14], we propose an extension of MTObject to multi-band images called M-MTObject, where the Max-Tree is computed on the best signal-to-noise ratio band but the component attributes and the statistical test use the information from all the bands.

In detail, M-MTObject firstly constructs the Max-Tree of the best signal-to-noise ratio band of the multi-band input image. The filtering strategy is the same as the MTObject statistical test, but node attributes are accumulated from all
<figure><img src="image_17.png" /><figcaption>(a) SDSS cutout band g.</figcaption></figure>

<figure><img src="image_18.png" /><figcaption>(b) CGO result on band (g,r).</figcaption></figure>

<figure><img src="image_19.png" /><figcaption>(c) MObject result on band g.</figcaption></figure>

<figure><img src="image_20.png" /><figcaption>(d) MObject result on band r.</figcaption></figure>

<caption>FIGURE 15: Experiment on a two-band SDSS image.</caption>

the bands. With this strategy, all the bands are forced to follow the selected Max-Tree, i.e., to follow the total order of the best signal-to-noise ratio band. This approach is simple, but we can see that each band has its own total order which likely disagrees with the total order of the selected band in some regions. These conflicted regions will introduce falsely significant nodes, leading to false positive detection.

## 4) Quantitative Result and Discussion

We compare CGO versus MObject/M-MObject on the FDS Simulation. Since the signal close to the border of the image is less reliable, we skip objects whose center is lying within 100 pixels from the borders. Precision and recall curves are presented in Fig. 14.

As can be seen from the curves, our proposed method CGO significantly improves on MObject/M-MObject on both precision and recall metrics in the FDS Simulation. All methods demonstrate robustness to the choice of hyper-parameter with favorable recalls (r > 0.7). As MObject is designed for single-band processing, M-MObject's performance on multi-band image could not surpass MObject on separated bands. Particularly, M-MObject's precision drops significantly compared to the other methods. This can be explained by the inconsistency between the single-band Max-Tree structure and the multi-band attributes. On the other hand, CGO efficiently leverages multi-band information, leading to better precision and recall compared to the state of the art in this simulation setting.

# D. EVALUATION ON REAL ASTRONOMICAL SURVEYS

We assess CGO versus MObject/M-MObject on real multi-band astronomical images SDSS, KiDS, and HST.

## 1) Real Datasets

We use three astronomical multi-band Surveys: the Sloan Digital Sky Survey (SDSS), the Kilo-Degree Survey (KiDS, [18]), and the Hubble Space Telescope Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey (HST

<figure><img src="image_21.png" /><figcaption>FIGURE 16: Evaluation on the KiDS-HST Dataset.</figcaption></figure>

CANDELS, [19]). The results of CGO and MObject on the real images are shown in Fig. 15.

## 2) Ground-truth

We used 100 image pairs, where each pair consisted of KiDS and HST CANDELS cutouts sharing the same field of view and centered on the same galaxies. All cutouts were taken from the same four source tiles: three KiDS tiles in u, g and r bands, and one HST CANDELS tile observed with the Advanced Camera for Surveys (ACS) in the F814W filter. All cutouts were located in RA range [53.0; 53.2] and DEC range [-27.9; -27.7] in the KiDS-South region of sky. Since HST CANDELS cutouts have much higher resolution and signal-to-noise ratio, we used the detection results obtained with MObject on these cutouts as the ground-truth for the KiDS images.

## 3) Metric

We use the same metrics as mentioned in Sec. V-C.

## 4) Quantitative Result and Discussion

We compare CGO and MObject/M-MObject [12] on the registered KiDS-HST images. As shown in Fig. 16, CGO achieves better F1-scores than MObject/M-MObject on this real dataset. Similarly to the previous simulation setting, M-MObject on multi-band images is again not better than MObject on single-band images, i.e., adding multi-band information confuses the object detector in the case of MObject. These results are consistent with the experiment performed in the FDS Simulation and with visual assessments. Note that all the F1-scores on the KiDS-HST experiment are much lower than the F1-scores in the FDS Simulation test. This is due to HST images (reference images) being much deeper than KiDS images, i.e., more objects are visible in the HST images, therefore many objects in the reference images are simply undetectable on the KiDS images. All in all, CGO outperforms the state of the art for source detection in the real KiDS-HST dataset.
## VI. CONCLUSION

We have explored how the component-graph structure can handle object detection on multi-band data. We have proposed Component-Graph Objects (CGO) – an object detection method along a set of novel node attributes on the component-graphs with application to multi-band astronomical images. Theoretically, our studies have shown that the component-graphs are better at preserving object structures compared to the classical component-trees. Practical experiments on both simulation and real astronomical surveys consistently confirm that CGO outperforms the state of the art on precision, recall, and F1-score metrics.

However, a current limitation of the proposed approach is its time complexity which may hamper the processing of large images. In future work, we plan to speed-up component-graph exploration with shape space filtering [10], [20].

## ACKNOWLEDGMENTS

This research was funded by the European Union Horizon 2020 program under the Marie Skłodowska-Curie grant agreement No. 721463 to the SUNDIAL ITN network and the Programme d'Investissements d'Avenir (LabEx BE-ZOUT ANR-10-LABX-58).

## REFERENCES

[1] P. Salembier and et al. Antiextensive connected operator for image and sequence processing. IEEE TIP, 7:555-570, 1998.

[2] E. J Breen and R. Jones. Attribute openings, thinnings, and granulometries. CVIU, 64:377-389, 1996.

[3] P. Monasse and F. Guichard. Fast computation of a contrast-invariant image representation. IEEE TIP, 9:860-872, 2000.

[4] Edwin Carlinet and Thierry Géraud. A comparative review of component tree computation algorithms. TIP, 23:3885-3895, 2014.

[5] T. Géraud, E. Carlinet, and et al. A quasi-linear algorithm to compute the tree of shapes of nd images. In ISMM, pages 98-110, 2013.

[6] E. Aptoula and S. Lefèvre. A comparative study on multivariate mathematical morphology. PR, 40:2914-2929, 2007.

[7] B. Perret and et al. Connected component trees for multivariate image processing and applications in astronomy. In ICPR, pages 4089-4092, 2010.

[8] N. Passat and B. Naegel. Component-trees and multivalued images: Structural properties. JMIV, 49:37-50, 2014.

[9] B. Naegel and N. Passat. Colour image filtering with component-graphs. In ICPR, pages 1621-1626, 2014.

[10] E. Grossiord, B. Naegel, and et al. Shape-based analysis on component-graphs for multivalued image processing. MMTA, 3:45-70, 2019.

[11] E. Bertin and S Arnouts. Sextractor: Software for source extraction. Astronomy and Astrophysics Supplement Series, 117:393-404, 1996.

[12] P. Teeninga, U. Moschini, S. C Trager, and et al. Statistical attribute filtering to detect faint extended astronomical sources. MMTA, 1, 2016.

[13] M. HF Wilkinson and et al. Sourcerer: A robust, multi-scale source extraction tool suitable for faint and diffuse objects. In IAU Symposium, volume 355, 2019.

[14] Caroline Haigh, Nushkia Chamba, Aku Venhola, Reynier Peletier, Lars Doorenbos, Matthew Watkins, and Michael HF Wilkinson. Optimising and comparing source-extraction tools using objective segmentation quality criteria. Astronomy & Astrophysics, 645:A107, 2021.

[15] T. Nguyen, Giovanni Chierchia, and et al. Cgo: Multiband astronomical source detection with component-graphs. In IEEE ICIP, 2020.

[16] T Yung Kong and Azriel Rosenfeld. Digital topology: Introduction and survey. Computer Vision, Graphics, and Image Processing, 48(3):357-393, 1989.

[17] A. Venhola. Evolution of dwarf galaxies in the Fornax cluster. PhD thesis, University of Groningen, 2019.

[18] K Kuijken, C Heymans, and et al. The fourth data release of the kilo-degree survey: ugri imaging and nine-band optical-ir photometry over 1000 square degrees. Astronomy & Astrophysics, 625:A2, 2019.

[19] Anton M Koekemoer, SM Faber, and et al. Candels: the cosmic assembly near-infrared deep extragalactic legacy survey—the hubble space telescope observations, imaging data products, and mosaics. The Astrophysical Journal Supplement Series, 197(2):36, 2011.

[20] Y. Xu, T. Géraud, and L. Najman. Connected filtering on tree-based shape-spaces. IEEE TPAMI, 38:1126-1140, 2015.
<figure><img src="image_22.png" /></figure>

THANH XUAN NGUYEN received the M.Sc. in computer science from JAIST, Japan in 2016. He is currently pursuing the Ph.D. in the same field at ESIEE Paris, Université Gustave Eiffel, France. His research interests include mathematical morphology, machine learning, and computer vision.

<figure><img src="image_23.png" /></figure>

GIOVANNI CHIERCHIA received the Engineering Degree in computer science from University Federico II of Naples, Italy in 2010, and the Ph.D. degree from Telecom ParisTech, France in 2015. From 2010 to 2011, he was a Research Engineer with Telecom ParisTech. Since 2015, he has been an Assistant Professor with ESIEE Paris, Université Gustave Eiffel, France. His research interests include optimization, machine learning, and image processing.

<figure><img src="image_24.png" /></figure>

OLEKSANDRA RAZIM received the M.Sc. in astronomy from V. N. Karazin Kharkiv National University, Ukraine, and is currently finishing her Ph.D. in the same field at the University of Naples 'Federico II' in Italy. Her research interests include extragalactic astrophysics, unsupervised machine learning and applications of machine learning to large astronomical datasets.

<figure><img src="image_25.png" /></figure>

REYNIER F. PELETIER (M.Sc. University of Leiden. Ph.D. University of Groningen, 1989) has been a postdoc at the Harvard-Smithsonian Center for Astrophysics, a postdoctoral fellow at the European Southern Observatory, and a postdoc in Groningen. In 1997 he obtained a PPARC Advanced Fellowship at the University of Durham, two years later he was appointed as lecturer at the University of Durham, and in 2003 he moved back to Groningen. He is currently a Full Professor at

the Kapteyn Astronomical Institute at the University of Groningen. He is interested in the evolution of galaxies, their dynamics and stellar populations, but also in instrumentation for large astronomical telescopes. He has edited several books and written more than 200 papers in refereed journals. At present, he is leading an EU International Training Network, named SUNDIAL, integrating computer scientists and astronomers to develop novel data analysis techniques for use in astronomy.

<figure><img src="image_26.png" /></figure>

LAURENT NAJMAN (SM'17) received the Habilitation à Diriger les Recherches in 2006 from University of Marne-la-Vallée, a Ph.D. of applied mathematics from Paris-Dauphine University in 1994 with the highest honor (Félicitations du Jury) and an "Ingénieur" degree from the Ecole des Mines de Paris in 1991. After earning his engineering degree, he worked in the central research laboratories of Thomson-CSF for three years, working on some problems of infrared image segmentation using mathematical morphology. He then joined a start-up company named Animation Science in 1995, as director of research and development. The technology of particle systems for computer graphics and scientific visualization, developed by the company under his technical leadership received several awards, including the "European Information Technology Prize 1997" awarded by the European Commission (Esprit programme) and by the European Council for Applied Science and Engineering and the "Hottest Products of the Year 1996" awarded by the Computer Graphics World journal. In 1998, he joined OCÉ Print Logic Technologies, as senior scientist. He worked there on various problem of image analysis dedicated to scanning and printing. In 2002, he joined the Computer Sciences Department of ESIEE Paris, where he is professor and a member of the Laboratoire d'Informatique Gaspard Monge, Université Gustave Eiffel. His current research interest is discrete mathematical morphology and discrete optimization.

<figure><img src="image_27.png" /></figure>

HUGUES TALBOT received the Habilitation from Université Paris-Est in 2013, the PhD from École des Mines de Paris in 1993 and the engineering degree from École Centrale de Paris in 1989. He was principal research scientist at CSIRO, Sydney, from 1994 to 2004. He was a professor at ESIEE Paris, Université Paris-Est, from 2004 to 2018. He is now a professor at CentraleSupélec, Université Paris-Saclay. He is the co-author or co-

editor of 7 books and over 200 articles in the area of mathematical morphology, discrete geometry, combinatorial and continuous optimization as well as machine learning and computer vision.

<figure><img src="image_28.png" /></figure>

BENJAMIN PERRET received his M.Sc. in Computer Science in 2007, and his Ph.D. in Image Processing in 2010 from the Université de Strasbourg (France). He currently holds a teacher-researcher position at ESIEE Paris, Université Gustave Eiffel, affiliated with the Laboratoire d'Informatique Gaspard-Monge. His current research interests include image processing and analysis.

...