# Measuring unit relevance and stability in hierarchical spatio-temporal clustering

Roy Cerqueti, Raffaele Mattera

<figure><img src="image_1.png" /></figure>

## To cite this version:

Roy Cerqueti, Raffaele Mattera. Measuring unit relevance and stability in hierarchical spatio-temporal clustering. Spatial Statistics, 2025, 66, pp.100880. 10.1016/j.spasta.2025.100880. hal-05109271

HAL Id: hal-05109271

https://univ-angers.hal.science/hal-05109271v1

Submitted on 12 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

ELSEVIER

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

Check for
updates

# Measuring unit relevance and stability in hierarchical spatio-temporal clustering

Roy Cerqueti a,b,*, Raffaele Mattera a,c

## ARTICLE INFO

**Keywords:**

Cluster analysis

Relevance assessment

Spatial clustering

Time series

Stability

## ABSTRACT

Understanding the significance of individual data points within clustering structures is critical to effective data analysis. Traditional stability methods, while valuable, often overlook the nuanced impact of individual units, particularly in spatial contexts. In this paper, we explore the concept of unit relevance in clustering analysis, emphasizing its importance in capturing the spatio-temporal nature of the clustering problem. We propose a simple measure of unit relevance, the Unit Relevance Index (URI), and define an overall measure of clustering stability based on the aggregation of computed URIs. Considering two experiments on real datasets with geo-referenced time series, we find that the use of spatial constraints in the clustering task yields more stable results. Therefore, the inclusion of the spatial dimension can be seen as a way to stabilize the clustering.

## 1. Introduction

Clustering algorithms play a central role in data analysis and pattern recognition, providing valuable insights into the inherent structures within data. However, the concept of unit relevance emerges as a fundamental yet often overlooked aspect. Unit relevance refers to the impact of individual data points on the clustering structure, specifically considering their inclusion or exclusion. It aims to answer the fundamental question of how the clustering changes when certain units are severely perturbed and/or removed.

Clustering stability methods traditionally rely on perturbations applied to the original dataset to assess the robustness of clustering algorithms to such shocks. Common approaches to measuring the stability of temporal partitions involve generating perturbations for each observation unit in the original dataset (Liu et al., 2022). He and Yu (2019) deals with the stability of an evolutionary K-means clustering procedure by considering different accuracy measures. Still on this topic, Mavroeidis and Marchiori (2014) discusses algorithms for practical feature selection to achieve a reliable balance between optimal prediction of the clustering exercise and reduction of uncertainty in the clustering. While these methods are effective in assessing overall stability, they may not capture the nuanced influence of individual units on the clustering structure, especially in scenarios that incorporate spatial dimensions. However, these methods predominantly assess stability at the aggregate level of entire clustering configurations, neglecting the importance of individual units within these clusters. In other words, stability is a property that a clustering algorithm may or may not have, while unit relevance is related to the statistical units involved in the clustering task. Our approach departs from this convention by focusing on the relevance of each unit to evaluate the stability of the cluster.

In the context of spatio-temporal data clustering (e.g. Benevento and Durante, 2024; Gaetan et al., 2024), the importance of individual units is enhanced by the presence of a spatial dimension. In this case, the proximity of units forms implicit links that

<footnote>ª Department of Social and Economic Sciences, Sapienza University of Rome, P.le A. Moro 5, Rome, I-00185, Italy</footnote>

<footnote>ᵇ GRANEM, University of Angers, SFR CONFLUENCES, F-49000, Angers, France</footnote>

<footnote>Department of Mathematics and Physics, University of Campania “Luigi Vanvitelli”, Viale Lincoln 5, Caserta, I-81100, Italy</footnote>

<footnote>* Corresponding author at: Department of Social and Economic Sciences, Sapienza University of Rome, P.le A. Moro 5, Rome, I-00185, Italy.</footnote>

<footnote>E-mail addresses: roy.cerqueti@uniroma1.it (R. Cerqueti), raffaele.mattera@uniroma1.it (R. Mattera).</footnote>
influence the clustering structure. Consider the analogy of a network: removing a highly connected node disrupts the network more than removing a peripheral node. In this case, the removal of a unit could indeed reshape the entire topology (Cerqueti et al., 2022; Moutsinas and Guo, 2020). For example, the centrality of a unit within its spatial context implies its relevance, whereas isolated units form singletons and have minimal relevance within the clustering framework. Thus, the spatial dimension not only emphasizes the interdependence between units, but also delineates their degree of relevance. Similar arguments can be applied to similarity in the time domain. However, it is not clear which type of clustering, temporal or spatio-temporal, guarantees more stable results.

From another point of view, the relevance of the individual units is critical in reliability theory and in game theory. In particular, the reference context is how the failure of one of the components of a system leads to the failure of the whole structure. When dealing with generalizations of standard *k*-out-of-*n* reliability systems – i.e., systems formed by *n* components that fail when *k* components fail (Birnbaum et al., 1961) – the relevance of the individual components is the most important issue to be explored both by theorists and for practical applications. On this, Wu and Chen (1994) introduces the weighted *k*-out-of-*n* systems with heterogeneous components and different roles leading to the collapse of the overall structure. Also, Cerqueti (2022) proposes a new class of *k*-out-of-*n* systems with components clustered on the basis of their relevance is proposed and studied. Barlow and Proschan (1975) provides a formal definition of component importance by introducing an indicator which explains how the failure of a given component can lead to the failure of the entire system.

For the latter context, the units importance is related to the role of a player in a coalition, when the players of a game can cooperate. We refer to the classical Shapley value, introduced by Lloyd Shapley in Shapley (1951). The grounding idea of the Shapley value is to determine the importance of a player by taking the mean of the difference between the payoffs of a coalition including the player and the same coalition with the exclusion of the considered player. Shapley value is at the core of a large strand of literature (e.g. Roth, 1988; Algaba et al., 2019, with a collection of several important contributions on the topic).

This paper adapts the theoretical foundation of Barlow and Proschan's importance index and Shapley value to clustering algorithms, aiming to assess each unit's relevance for clustering stability. A generic unit is relevant if its exclusion leads to a completely different partition of the remaining units, leading to a weakly stable clustering structure. In particular, after defining an appropriate measure of unit relevance, which we call the Unit Relevance Index (URI), we compare temporal and spatio-temporal clustering approaches in terms of the URI and the resulting clustering stability.

We consider the weighted Ward-like hierarchical clustering algorithm as discussed in Chavent et al. (2018) and Mattera and Franses (2023), where the temporal dimension is suitably weighted with the spatial dimension. The introduction of the spatial weight is used to measure how relevant the similarity in the spatial domain is compared to that in the temporal domain. This spatio-temporal clustering algorithm, considered also in other papers (e.g. Bucci et al., 2023), is a generalization of the hierarchical clustering to the spatio-temporal setting. Indeed, by constraining the spatial weight to be zero we get the standard (Ward Jr., 1963) hierarchical clustering. The generality of this algorithm ensures a direct comparability of unit relevance and stability between temporal and spatio-temporal clustering approaches.

The adopted spatio-temporal algorithm works as follows. As the classical Ward hierarchical clustering algorithm, the weighted spatio-temporal procedure of Chavent et al. (2018) starts with each data point in its cluster and then, step by step, it groups the points or clusters that are most alike. This is done in a way that keeps the differences within clusters small. As the process continues, the number of clusters decreases until eventually, all the data points are in one single cluster. Throughout, the goal is to maintain as much internal similarity within each cluster as possible, resulting in meaningful groupings that reflect the data structure.

We apply the procedure for computing the URI to two alternative datasets of different types. The first dataset concerns the clustering of Gross Domestic Product (GDP) of African countries, which is useful for the analysis of business cycle convergence (e.g. see Bénassy-Quéré and Coupet, 2005; Mattera and Franses, 2023). The degree of business cycle convergence is a key policy issue, especially when it comes to the feasibility of monetary unions. Indeed, business cycle synchronization is studied using clustering techniques, see Artis and Zhang (2002) and Ahlborn and Wortmann (2018) for European countries, Bénassy-Quéré and Coupet (2005) and Gammadigbe and Dioum (2022) for African countries and Quah and Crowley (2010) for countries in Asia. Overall, the literature relies on either cross-sectional or time series data, while Mattera and Franses (2023) recently proposed the use of a spatio-temporal method based on the gravity model.

The second dataset considers electricity consumption profiles in the region of Alberta, Canada. Indeed, cluster analysis is often applied to electricity demand data to identify patterns in consumption that can be useful for improving load forecasting (e.g. see Goia et al., 2010; Yilmaz et al., 2019). Electricity consumption curves are clustered either at the consumer level or at the provincial level (McLoughlin et al., 2015; Ferraro et al., 2016), depending on the type of forecasts to be produced. While the main approaches to identifying similar consumption curves are based on time series clustering, more recently spatio-temporal approaches have been proposed (e.g. see Sachs et al., 2019; Voulis et al., 2018).

The two datasets differ not only in their nature but also in their dimension. While the first dataset considers a short time series (annual data for fifty years), the second has a much larger temporal dimension (daily time series for three years). Moreover, the level of spatial aggregation is also different, as the first dataset contains data at the country level, while the second analysis is at the provincial level. These differences allow us to explore the calculation of unit relevance and the assessment of spatio-temporal clustering stability at different spatial resolutions.

The rest of the paper is structured as follows. Section 2 provides a detailed discussion of the methodology proposed for the measurement of unit relevance in the context of spatio-temporal clustering. In particular, Section 2.2 discusses the clustering framework considered in the paper – that is, spatio-temporal hierarchical clustering – and Section 2.3 introduces the Unit Relevance Index (URI). Section 3 assesses unit relevance and stability using two empirical cases. Section 3.1 discusses the application to business cycle synchronization, while Section 3.2 presents the results for the application to energy consumption time series. In both cases, spatio-temporal and time series clustering approaches are compared. Section 4 offers concluding remarks and future research directions.
## 2. Methodology

This section is devoted to the illustration of the proposed method. We start with a general overview of the statistical problem before entering the details.

Consider N units collected in a set $\mathcal{U} = \{u_1, ..., u_N\}$. We consider a distance measure $d: \mathcal{U} \times \mathcal{U} \to [0, +\infty)$ such that $d_{ij} := d(u_i, u_j)$ is the distance between units $u_i$ and $u_j$, for $i, j = 1, ..., N$. For simplicity, we refer hereafter to units through their indices, so that $j := u_j$, for each $j = 1, ..., N$. The set $\mathcal{U}$ is clustered in subsets on the basis of $d$. Specifically, we consider a set $C$ of $K$ subsets of $\mathcal{U}$, called clusters as $C = \{C_1, ..., C_K\}$. We assume that $C_h \cap C_k = \emptyset, \cup_{k=1}^K C_k = \mathcal{U}$ and that $u_i, u_j$ belong to the same cluster if and only if

$$d_{ij} \leq \max\{\min_h d_{ih}, \min_k d_{jk}\}. \quad (1)$$

Eq. (1) states that two units $u_i$ and $u_j$ are clustered together if their distance $d_{ij}$ is smaller than the distance of $u_i$ and $u_j$ from any other unit. In other words, the units $u_i$ and $u_j$ are clustered together if their distance is smaller than the one from their nearest neighbourhoods. Now, assume that the generic unit $u_i \in \mathcal{U}$ is removed from set $\mathcal{U}$. Then, the algorithm leads to a different clustering $C^{(-i)}$. The aim of our study is to explore the deviation in some sense between $C$ and $C^{(-i)}$, to assess the relevance of $u_i$ for the stability of the clustering. In particular, we define the unit $u_i$ as relevant if its exclusion leads to a completely different partition of the remaining units in $\mathcal{U}$. Therefore, according to our definition of relevance, if $u_i$ is relevant we find that the partitions $C$ and $C^{(-i)}$ significantly differ. In the opposite case, if the exclusion of a unit $u_i$ does not change the cluster structure substantially, we find $C$ and $C^{(-i)}$ to be very similar and say that $u_i$ is not relevant. We also highlight that when there is a high number of relevant units in a set $\mathcal{U}$, this leads to a less stable partition, in the sense that removing a unit change significantly the clustering structure.

In what follows, we focus on spatio-temporal data. Therefore, we assume that all the N units in the set $\mathcal{U}$ are georeferenced, which means that they are located in the space with known spatial coordinates. Moreover, we assume that an attribute over T ($t = 1, ..., T$) times is observed for each ith unit. Therefore we can write $x_{it}$ the value of the attribute x at time t for the georeferenced ith unit.

### 2.1. Hierarchical spatio-temporal clustering

The proposed clustering stability problem is based on the selection of a clustering algorithm. In this paper, we consider a recently introduced Ward-like spatio-temporal hierarchical clustering algorithm (Chavent et al., 2018; Mattera and Franses, 2023). In the standard Ward hierarchical clustering framework (Ward Jr., 1963), the distances $d_{ij}$ are computed in the attribute space of the set $\mathcal{U}$ defined above. In the context of time series data, the attribute space is represented by the observation of an attribute over T ($t = 1, ..., T$) periods (Aghabozorgi et al., 2015). When the units are georeferenced, the consideration of the attribute space alone is not sufficient for clustering, and both temporal and spatial dimensions should be considered together.

Let us consider a generic $C_k$ ($k = 1, ..., K$) cluster. Following the notation used in Chavent et al. (2018) and Mattera and Franses (2023), we define the spatio-temporal inertia of a generic $C_k$ cluster as the convex combination of the temporal inertia and the spatial cluster inertia

$$I(C_k) = (1 - \alpha) \sum_{i \in C_k} \sum_{j \in C_k} \frac{w_i w_j}{2 \sum_{i \in C_k} w_i} d_{ij,t}^2 + \alpha \sum_{i \in C_k} \sum_{j \in C_k} \frac{w_i w_j}{2 \sum_{i \in C_k} w_i} d_{ij,g}^2 \quad (2)$$

where $w_i$ is the weight associated with the ith spatial unit, usually set to $w_i = 1/N$, $d_{ij,t}^2$ is the squared distance between units i and j in (temporal) attribute space, and $d_{ij,g}^2$ is the squared distance in geographic space. The spatio-temporal inertia in Equation (2) is a convex combination of the temporal inertia and the spatial inertia, given by the spatial weight $\alpha$, which can be interpreted as the relevance of spatial information relative to temporal. The smaller the inertia $I(C_k)$ is, the more homogeneous are the units in the cluster $C_k$.

The construction of the partition follows a Ward-like hierarchical procedure where, starting from a partition with $K = N$ clusters of all singletons, two clusters are merged at each step until $K = 1$ and no further cluster can be merged. More generally, each step of the procedure involves merging two clusters such that a partition into K clusters is obtained from a previous one consisting of $K + 1$ clusters. The rule for merging two clusters is, for a fixed value of spatial weight $\alpha$, the minimization of the total spatio-temporal inertia, which is given by

$$W(C) = \sum_{k=1}^{K} I(C_k), \quad (3)$$

with $I(C_k)$ defined in Eq. (2). More in detail, the procedure starts with a partition equal to $\mathcal{U}$, i.e. formed by N distinct clusters of all singletons. Then, we merge two clusters {$i$} and {$j$} with minimum

$$\delta_{ij} = (1 - \alpha) \frac{w_i w_j}{w_i + w_j} d_{ij,t}^2 + \alpha \frac{w_i w_j}{w_i + w_j} d_{ij,g}^2 \quad (4)$$

which means merging the two most similar units. Hence, starting at the first step from a partition $C'$ with $K + 1 = N$ clusters we reduce to a new partition $C$ with $K = N - 1$ clusters, obtained by minimizing the total spatio-temporal inertia.

The clustering procedure then continues as follows. To get a new partition $C$ in $K$ clusters from a given partition $C'$ consisting of $K + 1$ clusters, we aggregate two clusters $A$ and $B$ from $C'$. The clusters $A$ and $B$ to be merged are those minimizing the total
<figure><img src="image_6.png" /><figcaption>Fig. 1. Automatic elbow method. The blue line indicates the total inertia in Eq. (3) under the K number of clusters in the x-axis. The dashed red line represents a baseline connecting the first (K = 1 in the figure) and last point (K = 8 in the figure) of the inertia curve, serving as a reference to locate the elbow. The elbow is detected as the point with the largest distance of the blue line from the straight red line. In this example, the elbow is detected for K = 4. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)</figcaption></figure>

inertia in Eq. (3), which are those minimizing the quantity

$$ \delta(\mathcal{A}, \mathcal{B}) = I(\mathcal{A} \cup \mathcal{B}) - I(\mathcal{A}) - I(\mathcal{B}). \qquad (5) $$

This is because, given the Eq. (3), after merging A with B the new partition C has total inertia given by

$$ W(C) = W(C') - I(A) - I(B) + I(A \cup B), $$

which means that the new partition C has a total inertia similar to that of C', but where the within cluster inertia of the merged cluster (A ∪ B) substitutes those associated with A and B. Thus, given that W(C') is fixed, the minimum within-cluster inertia of partition C obtained in the iteration is defined as

$$ W(C) = \min_{A, B \in C'} \delta(\mathcal{A}, \mathcal{B}), \qquad (6) $$

to ensure that W(C) < W(C'). This process is repeated until K = 1 and all N units belong to the same cluster. In this last step, no further merging is possible and the algorithm stops.

To detect the optimal number of clusters K, we apply an automatic elbow detection method (e.g. see Morgado et al., 2023). The scree-plot is constructed by plotting the total spatio-temporal inertia against the number of clusters K. Fig. 1 shows a scree-plot example. The y-axis represents the total spatio-temporal inertia as in Eq. (3), while the x-axis shows the number of clusters. The scree-plot begins with the total inertia when all data points belong to a single cluster, which results in a high inertia value. This is the point with spatio-temporal inertia equal to 100 for K = 1 in Fig. 1. As additional clusters are considered, the inertia decreases. Indeed, the inertia decreases as the number of clusters increases, indicating tighter groupings as more clusters are formed. Beyond a certain point, that we identify as the elbow, the reduction in inertia slows down, meaning that considering more clusters does not significantly improve the partition quality. A straight red dashed line is drawn from the point where K = 1 (the highest inertia) to the point Kmax = 8. The elbow is the point on the blue scree plot with the largest distance from the red line. In the example in Fig. 1, the elbow is located at K = 4.

The described procedure assumes a fixed value of the spatial weight α. To choose α, we adopt the heuristic procedure proposed by Chavent et al. (2018), which finds the best compromise between the loss in temporal attribute space and the loss in geographical homogeneity. To be more precise, let us define $W_t(C^α) = \sum_{k=1}^K \sum_{i \in C_k} \sum_{j \in C_k} d_{ij,t}$ the total within-cluster inertia considering the distances in the temporal attribute space, with the K clusters obtained with the spatio-temporal hierarchical algorithm for a given value α. We similarly define $W_g(C^α) = \sum_{k=1}^K \sum_{i \in C_k} \sum_{j \in C_k} d_{ij,g}$ as the total within-cluster inertia considering the distances in terms of spatial dimension. Then, we define $W_t(P_1) = \sum_i \sum_j d_{ij,t}$ and $W_g(P_1) = \sum_i \sum_j d_{ij,g}$ the total inertia associated with temporal attribute and spatial distances, respectively. We compare the proportion of the total inertia explained by the partition obtained with the temporal dimension

$$ Q_t(C^\alpha) = 1 - \frac{W_t(C^\alpha)}{W_t(P_1)}, $$

with the proportion associated with the spatial dimension

$$ Q_g(C^\alpha) = 1 - \frac{W_g(C^\alpha)}{W_g(P_1)}, $$

for different S values of α. For example, we can consider $α = \{α_1 = 0, α_2 = 0.01, ..., α_{S-1} = 0.99, α_S = 1\}$. The difference between the two criteria for all the S values of α provides a simple way for choosing the mixing parameter, that is, when the difference between
<figure><img src="image_7.png" /><figcaption>Fig. 2. Criterion for choosing optimal α: (a) shows the intersection of $Q_t(C^α)$ and $Q_g(C^α)$, while (b) shows the squared difference between $Q_t(C^α)$ and $Q_g(C^α)$, highlighting the minimum at $α = 0.5$.</figcaption></figure>

the loss of temporal homogeneity and larger geographical cohesion, obtained through increasing values of $α$, is minimum. Fig. 2 shows a visualization of the approach adopted for the spatial weight $α$ selection. In particular, Fig. 2(a) shows the intersection of the two curves, which is equivalent to minimizing the difference between the two as shown in Fig. 2(b).

## 2.2. Temporal and spatial distance measures

Evidently, several choices for the distance measures can be implemented. In line with the feature-based approaches – that aim at clustering time series with similar characteristics, such as pairwise correlation, auto-correlation function, distribution parameters and many others (e.g. see Caiado et al., 2006; Lubba et al., 2019; Wang et al., 2006) – we here consider for the temporal dimension a correlation-based distance between the time series (Caiado et al., 2006), which is given by

$$d_{ij,t} = \sqrt{2(1 - \rho_{ij})}, \qquad (7)$$

where $ρ_{ij}$ is the sample pairwise Pearson correlation coefficients of the temporal patterns of two units $i$ and $j$. More precisely, if we define $x_{it}$ ($x_{jt}$) as the value of the attribute $x$ for the $i$th ($j$th) unit at time $t$, the correlation coefficient is computed as

$$\rho_{ij} = \frac{[\sum_{t=1}^{T} (x_{it} - \bar{x}_i)(x_{jt} - \bar{x}_j)]}{\sqrt{[\sum_{t=1}^{T} (x_{it} - \bar{x}_i)^2][\sum_{t=1}^{T} (x_{jt} - \bar{x}_j)^2]}} \qquad (8)$$

For spatial similarity, we notice that observations located close to each other in geographical space may have similar characteristics, and therefore spatial clustering can serve as a tool to instrument similarities in terms of unobserved characteristics at the spatial level. Spatial clustering can be achieved in different ways, depending mainly on the measure used to quantify proximity. In this context, we say that units $i$ and $j$ are contiguous if they are geographically adjacent, i.e. they share a common border. We introduce a square symmetric matrix of order $N$, denoted $P$. The diagonal elements of $P$ are set to zero, while the off-diagonal elements, denoted by

$$p_{ij} = \begin{cases} 1 & \text{if unit } i \text{ is contiguous to unit } j. \\ 0 & \text{otherwise.} \end{cases}$$

A suitable distance between $i$ and $j$ is given by Chavent et al. (2018)

$$d_{ij,g} = 1 - p_{ij}, \qquad (9)$$

which takes the value 0 if $i$ and $j$ are contiguous and 1 otherwise. The use of Equation (9), once considered in a given clustering algorithm, ensures that statistical units that are closer in geographical space are clustered together. It is not the only way to approach the spatial dimension in clustering. Important properties such as spatial dependence and heterogeneity over space are only partially taken into account in these distances (see e.g. Fouedjio, 2016; Mattera, 2022). Nevertheless, as shown in Chavent et al. (2018), when combined with another non-spatial distance measure, it can be successfully used to provide a soft spatial constraint on the final partition.
<figure><img src="image_8.png" /><figcaption>Fig. 3. Flowchart illustrating the procedure adopted for assessing unit relevance in clustering. Starting from the collection $U$, we define $N$ alternative partitions $C^{(-1)}, \dots, C^{(-i)}, \dots, C^{(-N)}$ where the individual units are taken alternatively and considered as singletons. We then compute the URI indices $\gamma_1, \dots, \gamma_i, \dots, \gamma_N$.</figcaption></figure>

## 2.3. Measuring individual unit relevance

The hierarchical spatio-temporal clustering algorithm provides a suitable approach for finding subsets of $U$, i.e. $C = \{C_1, \dots, C_K\}$, considering both spatial and temporal similarity. Now, let us consider another clustering of $U$, namely $C' = \{C'_1, \dots, C'_{K'}\}$. To study the relevance of individual unit $u_i$, we introduce a measure $\rho(C, C')$ based on the deviation between $C$ and $C'$.

Specifically, we define $C^{(-i)}$ as the clustering obtained by imposing that unit $u_i$ is relegated in a singleton cluster. In doing so, $C$ and $C' = C^{(-i)}$ are clustering of the same set $U$, but we are implicitly removing $u_i$ from the clustering exercise when constructing $C^{(-i)}$. To be more precise, while $C'$ indicates a generic alternative clustering of $U$, the notation $C^{(-i)}$ indicates a specific type of alternative partition, that is the one obtained when the $i$th unit is constrained to form a singleton cluster.

Inspired by the stability literature, an appropriate measure of relevance can be computed by comparing the set $C^{(-i)}$ partitions according to some external index measure of the agreement between partitions, such as the Rand index (Rand, 1971). Let us define $N_{11}$ as the number of spatial units that are in the same cluster in both $C$ and $C^{(-i)}$, $N_{00}$ as those that are in different clusters in both $C$ and $C^{(-i)}$, $N_{01}$ those that are in the same cluster for $C$ but in different clusters for $C^{(-i)}$, and $N_{10}$ the number of units that are in different clusters for $C$ but in the same cluster for $C^{(-i)}$. In this setting, $N_{11}$ and $N_{00}$ can be used as measures of the degree of agreement between $C$ and $C^{(-i)}$, while conversely $N_{01}$ and $N_{10}$ can be seen as measures of disagreement. Note that $N_{00}$, $N_{01}$, $N_{10}$ and $N_{11}$ depend on $i$. Such dependence is conveniently omitted for simplicity. Then we could define

$$ \tilde{\gamma}_i = \frac{(N_{00} + N_{11})}{\binom{N}{2}}, \qquad (10) $$

which lies within the range of [0.5,1]. We note that values $N_{00}$, $N_{11}$, $N_{01}$ and $N_{10}$ could be represented in a format similar to a confusion matrix, though their interpretation differs from that in a standard confusion matrix. In a traditional confusion matrix, indeed, the values represent counts of individual assignments as correct or incorrect for each class. By contrast, for the Eq. (10) these values capture pairwise agreement (or disagreement) between two partitions: $N_{00}$ and $N_{11}$ measure the number of point pairs that are clustered together or separately in both partitions, while $N_{01}$ and $N_{10}$ represent cases where one partition groups points together while the other separates them. Thus, Eq. (10) assesses similarity at the pair level between two partitions, rather than individual classification accuracy. The value of $\tilde{\gamma}_i$ measures the degree of overlap between the original partition $C$ and the new partition $C' = C^{(-i)}$. Therefore, a large value of $\tilde{\gamma}_i$ suggests that removing the $i$th unit does not change the original partition $C$. Vice-versa, a low value of $\tilde{\gamma}_i$ suggests that there is a lack of overlap between the original partition and the new one. In other words, if $\tilde{\gamma}_i$ is low then the $i$th unit is relevant, as its exclusion modifies considerably the original partition.

Hubert and Arabie (1985) proposed an adjusted version to the Eq. (10) that is always between the range [0,1], that is

$$ \gamma_i = \frac{2(N_{00}N_{11} - N_{01}N_{10})}{(N_{00} + N_{01})(N_{01} + N_{11}) + (N_{00} + N_{10})(N_{10} + N_{11})}. \qquad (11) $$

We call the measure $\gamma_i$ in Equation (11) the Unit Relevance Index (URI) of the unit $u_i$. The larger the value of the URI, the greater the relevance of $u_i$. This happens because moving the $i$th unit to a singleton significantly changes the clustering structure.

Fig. 3 gives an overview of the proposed approach for measuring unit relevance. Let us assume that we are interested in evaluating the unit relevance and stability of the hierarchical spatio-temporal clustering algorithm described in Section 2.1. Given the set of units $U$, we obtain a first partition $C$. Then, we obtain a first alternative partition, called $C^{(-1)}$ that is obtained considering the unit $u_1$ as a singleton and cluster the remaining $N-1$ units with the hierarchical spatio-temporal clustering algorithm under evaluation. By comparing the overlap of the partitions $C$ and $C^{(-1)}$, we compute the index $\gamma_1$ as described in Eq. (11).
<figure><img src="image_9.png" /><figcaption>Fig. 4. Estimated business cycles of the N = 46 continental African countries. The Hodrick-Prescott filter with λ = 6.25 has been used for extracting the cyclical component in each series. The correlation-based distance, shown in Eq. (7), is applied to these series to measure the temporal distance of the African business cycles.</figcaption></figure>

In Fig. 3 is represented by the arrows from the boxes C and C<sup>(-1)</sup> to the box of γ<sub>1</sub>. Then, we compare the original partition C with another alternative one, called C<sup>(-2)</sup> where the unit u<sub>2</sub> is considered as a singleton and the remaining N - 1 units are clustered. From the comparison of the overlap between C and C<sup>(-2)</sup> we compute γ<sub>2</sub>. Fig. 3 repeat this operation for all the N units included in the set V, so that the unit relevance index for each i-th unit is computed. The last comparison is made between C and C<sup>(-N)</sup>, where the last unit u<sub>N</sub> is considered as a singleton and the remaining N - 1 units are clustered. The resulting URI is called γ<sub>N</sub>.

In the end, we can define an overall measure of cluster stability based on the aggregated measure

$$
\bar{\gamma}(C) = \frac{1}{N} \sum_{i=1}^{N} \gamma_i. \qquad (12)
$$

As argued in Hennig (2007), other aggregation measures could be adopted, such as the median or specific quantiles. However, the mean seems to be the most appropriate measure to adopt, especially in the context of unit relevance. By using the median or the 4th quantile, for example, we would make our measure of stability depend too much on the relevance of specific units located in a few points of the empirical distribution, which is inconvenient.

The definition of γᵢ in Equation (11) – which is based on the comparison between C and C⁽⁻¹⁾ – assures that γᵢ is a modified version of the Barlow–Proschan important index (see Barlow and Proschan, 1975) and the Shapley value (see Shapley, 1951), suitably adapted to clustering procedures.

3. Applications to real data

In what follows, we discuss two illustrative examples of African business cycles and the daily load consumption in Canada. In both applications, we adopt the distance in Eq. (7) to measure the temporal distance and Eq. (9) for the spatial distance. We notice that the correlation in cyclical components – therefore assuming that xᵢₜ is a proxy variable for the economic activity of the i-th unit over time – is often used for measuring the similarity of business cycles in economics literature (e.g. see Ahlborn and Wortmann, 2018; Papageorgiou et al., 2010), but it is also widely adopted for clustering similar load consumption profiles (Rajabi et al., 2020).

For each of the two experiments with real data, we allow the detection of a number of clusters K' ≠ K in C⁽⁻¹⁾. The relevance of the unit also accounts for the incorrect choice of the number of clusters, since we assume that a unit is more relevant if its exclusion causes a change in the detected K'.

3.1. Clustering african business cycles

The first illustrative example is based on the clustering of African countries with similar business cycles. We consider the yearly GDP time series in the period 1960–2016, reconstructed by Franses and Vasilev (2019), of continental African counties, thus by excluding Islands and countries with no data available, so that the final sample consists of N = 46 statistical units (see Fig. 4). We estimate the cyclical components from each GDP time series using the (Hodrick and Prescott, 1997) filter, choosing a value of the smoothing parameter λ = 6.25 as suggested by previous studies (e.g. see Ravn and Uhlig, 2002; De Jong and Sakarya, 2016).

Unlike Franses (2020) and Mattera and Franses (2023), which use the feature DTW-based distance to measure African business cycle similarity, we follow other studies on business cycle synchronization and use a correlation-based approach in computing the distances between countries, so that countries with a highly correlated cyclical component are clustered together. The spatial dimension is used to instrument unobserved spatial effects and supports the gravity model in economics.
<figure><img src="image_10.png" /><figcaption>Fig. 5. Selection of the spatial weight α for the spatio-temporal clustering, for the Africa business cycle dataset. As explained in Section 2.1, the value of α is chosen such that the difference between the two curves is minimized (see also Fig. 2(b)). D0 and D1 represent the temporal and spatial distance measures, respectively.</figcaption></figure>

<figure><img src="image_11.png" /></figure>

(a) Scree-plot used for the automatic elbow detection method. As in Figure 1, the elbow is detected at K = 3, that is the point with the largest distance from the solid line to the dashed one.

(b) Map of Africa showing the cluster structure (K = 3) obtained with the hierarchical spatio-temporal clustering procedure.

<caption>Fig. 6. Results obtained with the spatio-temporal hierarchical clustering as discussed in Section 2.1. Figure (a) shows the scree-plot, and Figure (b) the map of Africa with the resulting cluster structure.</caption>

We then apply the hierarchical spatio-temporal clustering algorithm to the data. First, we choose the optimal spatial weight α (see Fig. 5). As explained in Section 2.1, the value of α is chosen such that the difference between the two curves is minimized. Therefore, the best compromise is obtained by setting α = 0.2.

Then, we choose the number of clusters with the elbow criterion. Fig. 14 shows the result of the number of clusters selection and the resulting baseline partition. In particular, Fig. 6(a) suggests a partition into K = 3 clusters, given the spatial weight α = 0.2. The obtained partition, shown in Fig. 6(b), is the following. Cluster 1 includes most of the central African countries, while Cluster 2 resembles the Common Monetary Union area given by the countries in the south of Africa. In the end, Cluster 3 highlights the presence of a cluster in East Africa.

Starting from the obtained baseline partition in Fig. 6(b), we aim to study the unit relevance from the clustering viewpoint. We follow the procedure highlighted in Fig. 3, thus excluding one country at time and relegating it as a singleton cluster. We then evaluate the agreement between the partitions according to Equation (11). The spatial distribution of the estimated URI is shown in Fig. 7. Fig. 7 shows that most of the units have quite high relevance, as their removal from the main clustering structure changes much of the underlying clustering structure. About 80% of the countries have a URI less than 0.5. Botswana and Equatorial Guinea are the least relevant in terms of URI, and indeed these have been recognized as outliers in terms of GDP growth (Franses, 2020). Other countries with low relevance are Angola, Cameroon, Central African Republic, Chad, Congo and Egypt. We note that most of these countries belong to cluster 1, suggesting that other countries in this cluster seem to cluster together even if one of the above countries is removed.
<figure><img src="image_12.png" /><figcaption>**Fig. 7.** Map of Africa reporting the unit relevance index (URI) for the considered countries in the datasets. The lower the value, the higher the relevance. The map is produced with the URI computed using the hierarchical spatio-temporal clustering approach discussed in Section 2.1.</figcaption></figure>

<figure><img src="image_13.png" /></figure>

(a) Scree-plot used for the automatic elbow detection method. As in Figure 1, the elbow is detected at $K = 3$, that is the point with the largest distance from the solid line to the dashed one.

<figure><img src="image_14.png" /></figure>

(b) Map of Africa showing the cluster structure ($K = 3$) obtained with the hierarchical temporal clustering procedure.

<caption>**Fig. 8.** Results obtained with the temporal hierarchical clustering, that is the procedure presented Section 2.1 with the constraint $\alpha = 0$. Figure (a) shows the scree-plot, and Figure (b) the map of Africa with the resulting cluster structure.</caption>

Next, we compare the results of the spatio-temporal clustering with those obtained using the temporal approach alone and also evaluate the difference, in terms of unit relevance, between the two obtained clustering. For hierarchical time series clustering, we adopt the algorithm discussed in Section 2 with the constraint $\alpha = 0$. The results are shown in Fig. 8, where Fig. 8(a) shows the elbow detection for the number of clusters *K* and Fig. 8(b) shows the map of Africa with the resulting cluster structure into $K = 3$ clusters. Compared to the previous partition with spatio-temporal approach, we find that both approaches cluster together in southern African countries, which is consistent with previous research showing that these countries have synchronized business cycles. However, the composition of the remaining two clusters is very different.

We then compute the URI for the units considering the case of time series clustering. As in the spatio-temporal case shown in Fig. 6(b), Fig. 8(b) shows the spatial distribution of the URI. We find that the relevance is very high for most of the units, and that the URI values are generally lower compared to the spatio-temporal clustering. Lower URI values result in a less stable partition. In the temporal case, almost 90% of countries in the sample have a URI value lower than 0.5, which is indicated with the green colour in Fig. 8(b). This highlights that almost all units in the sample are of high relevance, thus suggesting that the inclusion of spatial constraint improves the robustness of the clustering results. Botswana is the unit with the lowest relevance also in this experiment, thus confirming that this probably happens because it is a noisy unit. Therefore, considering Botswana as a singleton does not modify the overall clustering structure, and this explains its high URI value. Interestingly, Equatorial Guinea, the other known outlier in
<figure><img src="image_15.png" /><figcaption>**Fig. 9.** Map of Africa reporting the unit relevance index (URI) for the considered countries in the datasets. The lower the value, the higher the relevance. The map is produced with the URI computed using the hierarchical temporal clustering approach, using the procedure discussed in Section 2.1 with the constraint $\alpha = 0$, i.e. removing the spatial component.</figcaption></figure>

the sample, has a high URI value compared to other units in the sample. The other less relevant units are, according to the time series partition, Algeria, Angola, Benin, Burundi and Congo (see Fig. 9).

The overall measure of stability based on units relevance Equation (12) takes value $\bar{\gamma}(C) = 0.29$, for the spatio-temporal clustering, while it is equal to $\bar{\gamma}(C) = 0.22$ for the time series clustering. This result suggests that overall the spatio-temporal hierarchical clustering solution turns out to be much more stable than its temporal counterpart. In other words, it seems that the inclusion of a spatial constraint guarantees a more stable clustering result.

## 3.2. Clustering daily load consumption in Alberta, Canada

The second illustrative example is based on load consumption data in the areas in Alberta (Canada) as defined by the AESO. We consider data from 1 May 2020 to 31 October 2023¹, which are sampled at hourly frequency. For clustering purposes, we consider daily aggregated load consumption time series (Ferraro et al., 2016; Sachs et al., 2019). We have the total energy consumption in each of the $N = 42$ AESO areas for $T = 1279$ days. [The map of Alberta with AESO areas is shown in Fig. 10(a)], while the time series are shown in Fig. 10(b).

We notice that the time series scales differ within the considered spatial areas. Therefore, we normalize the time series through index number, that is, dividing the time series at time $t$ by its value at the first point in time $t = t0$ such that $z_{it} = x_{it}/x_{it0}$. In doing so, we let the $N$ units having series in the same scale. Before computing URI for the AESO areas, we first apply the hierarchical spatio-temporal clustering algorithm to the data. Fig. 11 shows the selection of the optimal spatial weight $\alpha$, suggesting that the best compromise between temporal and spatial dimension is achieved when $\alpha = 0.3$.

Next, we choose the number of clusters with the elbow criterion, obtaining a distinction into $K = 5$ clusters as shown in Fig. 12(a). The resulting partition of AESO areas is shown in Fig. 12(b). We notice that Cluster 2 is less numerous compared to the others. In particular, Cluster 4 is the largest in terms of size.

We follow again the procedure highlighted in Fig. 3, thus excluding one AESO area at the time and relegating it as a singleton cluster and evaluating the agreement between the partitions according to the measure defined in Equation (11). The results are shown in Fig. 13, which shows the spatial distribution of the URI for the entire Alberta region. In this experiment, we find that 66% of units are relevant, that is showing URI lower than 0.5. Therefore, we again find that most of the units are relevant in determining the clustering structure. Interestingly, we find that the units in Cluster 5 are those with the lowest URI value, which is on average equal to 0.06. Indeed, it seems that the spatio-temporal algorithm isolates in a separate cluster the AESO areas with the largest relevance.

We then compare the results of the spatio-temporal clustering with those obtained using the temporal approach alone and also evaluate the difference. Fig. 14(a) shows that in this case the elbow is detected at $K = 3$. The resulting partition is shown in Fig. 14(b). The time series clustering approach detects a small Cluster 2 and a large Cluster 1, whose size is much larger than both Cluster 2 and Cluster 3. The overlap of this partition with the one obtained using the spatio-temporal approach (Fig. 12(b)) is very low.

<footnote n="1">¹ Data is available at the following link: https://www.aeso.ca/market/market-and-system-reporting/data-requests/hourly-load-by-area-and-region</footnote>
<figure><img src="image_16.png" /><figcaption>**Fig. 10.** Overview of Alberta region (Canada) with corresponding AESO areas and their daily load consumption. AESO areas are highlighted in different colours.</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>**Fig. 11.** Selection of the spatial weight $\alpha$ for the spatio-temporal clustering for the Canada load consumption dataset. As explained in Section 2.1, the value of $\alpha$ is chosen such that the difference between the two curves is minimized (see also Fig. 2(b)). D0 and D1 represent the temporal and spatial distance measures, respectively.</figcaption></figure>

Fig. 15 shows the spatial distribution of the calculated unit relevance indices for the time series clustering approach. Compared to Fig. 13, in Figure 15 we find a larger presence of units with low URI values, that is, units that are relevant for the cluster structure. Indeed, by evaluating the differences between both approaches in terms of the overall measure of stability Equation (12), we find that it takes value $\bar{\gamma}(C) = 0.32$, for the spatio-temporal clustering and $\bar{\gamma}(C) = 0.28$ for the time series clustering. Therefore, we find again that the inclusion of a spatial constraint allows for a more stable clustering result.

## 4. Conclusions

In this paper we have addressed the often overlooked aspect of unit relevance in clustering algorithms, particularly in the context of spatio-temporal clustering. By introducing the Unit Relevance Index (URI) – which is an adaption to clustering of the ideas behind well-known relevance index, like the importance index by Barlow and Proschan and the Shapley value – we quantified the importance of individual units in assessing clustering stability.

Our comparative analysis between temporal and spatio-temporal clustering approaches, using two different datasets, provided valuable insights into the differences in clustering stability and unit relevance at different spatial resolutions. We examined the
<figure><img src="image_18.png" /></figure>

(a) Scree-plot used for the automatic elbow detection method. As in Figure 1, the elbow is detected at $K = 5$, that is the point with the largest distance from the solid line to the dashed one.

<figure><img src="image_19.png" /></figure>

(b) Map of Alberta region (Canada) showing the cluster structure ($K = 5$) obtained with the hierarchical spatio-temporal clustering procedure.

<figure><img src="image_20.png" /><figcaption>**Fig. 13.** Map of Alberta region (Canada) reporting the unit relevance index (URI) for the considered areas in the datasets. The lower the value, the higher the relevance. The map is produced with the URI computed using the hierarchical spatio-temporal clustering approach, using the procedure discussed in Section 2.1.</figcaption></figure>

convergence of business cycles in Africa and electricity consumption profiles in Alberta, Canada, using both temporal and spatio-temporal clustering techniques. The results highlight the importance of considering unit relevance, especially in scenarios with spatial dimensions, where the proximity of units affects the clustering structure. We found that in both empirical cases considered, the spatio-temporal approach consistently provided more stable clustering than the temporal approach. This suggests that incorporating spatial information improves the stability and robustness of clustering algorithms, especially in scenarios with implicit links between data points.

While this study sheds light on the relevance of individual entities in the assessment of spatio-temporal clustering stability, there are several avenues for future research. For example, extending the analysis to different types of temporal and spatial distance measures and algorithms could help to generalize our findings. Investigating the impact of unit relevance in dynamic clustering scenarios where the data evolves could provide a further understanding of the dynamics of clustering stability.

In conclusion, our study highlights the importance of considering unit relevance in clustering algorithms and provides a foundation for future research aimed at advancing the understanding and application of clustering techniques in diverse domains.
<figure><img src="image_21.png" /></figure>

(a) Scree-plot used for the automatic
elbow detection method. As in Figure
1, the elbow is detected at K = 3,
that is the point with the largest
distance from the solid line to the
dashed one.

(b) Map of Alberta region (Canada) showing the
cluster structure (K = 3) obtained with the hierar-
chical temporal clustering procedure.

<figure><img src="image_22.png" /><figcaption>Fig. 15. Map of Alberta region (Canada) reporting the unit relevance index (URI) for the considered areas in the datasets. The lower the value, the higher the relevance. The map is produced with the URI computed using the hierarchical temporal clustering approach, using the procedure discussed in Section 2.1 with the constraint α = 0, i.e. removing the spatial component.</figcaption></figure>

References

Aghabozorgi, S., Shirkhorshidi, A.S., Wah, T.Y., 2015. Time-series clustering-a decade review. Inf. Syst. 53, 16-38.

Ahlborn, M., Wortmann, M., 2018. The core-periphery pattern of European business cycles: A fuzzy clustering approach. J. Macroecon. 55, 12-27.

Algaba, E., Fragnelli, V., Sánchez-Soriano, J., 2019. Handbook of the Shapley value. CRC Press, Boca Raton, FL.

Artis, M.J., Zhang, W., 2002. Membership of EMU: A fuzzy clustering analysis of alternative criteria. J. Econ. Integr. 17 (1), 54-79.

Barlow, R.E., Proschan, F., 1975. Importance of system components and fault tree events. Stochastic Process. Appl. 3 (2), 153-173.

Bénassy-Quéré, A., Coupet, M., 2005. On the adequacy of monetary arrangements in sub-Saharan Africa. World Econ. 28 (3), 349-373.

Benevento, A., Durante, F., 2024. Correlation-based hierarchical clustering of time series with spatial constraints. Spatial Stat. 59, 100797.

Birnbaum, Z.W., Esary, J.D., Saunders, S.C., 1961. Multi-component systems and structures and their reliability. Technometrics 3 (1), 55-77.

Bucci, A., Ippoliti, L., Valentini, P., et al., 2023. Analysing spatiotemporal patterns of COVID-19 confirmed deaths at the NUTS-2 regional level. Reg. Stat. 13 (2), 214-239.

Caiado, J., Crato, N., Peña, D., 2006. A periodogram-based metric for time series classification. Comput. Statist. Data Anal. 50 (10), 2668-2684.

Cerqueti, R., 2022. A new concept of reliability system and applications in finance. Ann. Oper. Res. 312 (1), 45-64.

Cerqueti, R., Ciciretti, R., Dalò, A., Nicolosi, M., 2022. A new measure of the resilience for networks of funds with applications to socially responsible investments.
Phys. A 593, 126976.

Chavent, M., Kuentz-Simonet, V., Labenne, A., Saracco, J., 2018. ClustGeo: an R package for hierarchical clustering with spatial constraints. Comput. Statist. 33 (4), 1799-1822.
De Jong, R.M., Sakarya, N., 2016. The econometrics of the Hodrick-Prescott filter. Rev. Econ. Stat. 98 (2), 310-317.

Ferraro, P., Crisostomi, E., Tucci, M., Raugi, M., 2016. Comparison and clustering analysis of the daily electrical load in eight European countries. Electr. Power Syst. Res. 141, 114-123.

Fouedjio, F., 2016. A hierarchical clustering method for multivariate geostatistical data. Spatial Stat. 18, 333-351.

Franses, P.H., 2020. Do African economies grow similarly? Cybern. Syst. 51 (8), 746-756.

Franses, P.H., Vasilev, S., 2019. Real GDP growth in Africa, 1963-2016. Technical Report, Econometric Institute, Rotterdam.

Gaetan, C., Girardi, P., Musau, V.M., 2024. Spatial quantile clustering of climate data. Adv. Data Anal. Classif. 1-29.

Gammadigbe, V., Dioum, S.B., 2022. Monetary integration in West Africa: Are business cycles converging? Afr. Dev. Rev. 34 (1), 68-80.

Goia, A., May, C., Fusai, G., 2010. Functional clustering and linear regression for peak load forecasting. Int. J. Forecast. 26 (4), 700-711.

He, Z., Yu, C., 2019. Clustering stability-based evolutionary k-means. Soft Comput. 23 (1), 305-321.

Hennig, C., 2007. Cluster-wise assessment of cluster stability. Comput. Statist. Data Anal. 52 (1), 258-271.

Hodrick, R.J., Prescott, E.C., 1997. Postwar US business cycles: an empirical investigation. J. Money, Credit., Bank. 1-16.

Hubert, L., Arabie, P., 1985. Comparing partitions. J. Classification 2, 193-218.

Liu, T., Yu, H., Blair, R.H., 2022. Stability estimation for unsupervised clustering: A review. Wiley Interdiscip. Rev. Comput. Stat. 14 (6), e1575.

Lubba, C.H., Sethi, S.S., Knaute, P., Schultz, S.R., Fulcher, B.D., Jones, N.S., 2019. Catch22: Canonical time-series CHaracteristics: Selected through highly comparative time-series analysis. Data Min. Knowl. Discov. 33 (6), 1821-1852.

Mattera, R., 2022. A weighted approach for spatio-temporal clustering of COVID-19 spread in Italy. Spatial Spatio-Temporal Epidemiol. 41, 100500.

Mattera, R., Franses, P.H., 2023. Are African business cycles synchronized? Evidence from spatio-temporal modeling. Econ. Model. 128, 106485.

Mavroeidis, D., Marchiori, E., 2014. Feature selection for k-means clustering stability: theoretical analysis and an algorithm. Data Min. Knowl. Discov. 28, 918-960.

McLoughlin, F., Duffy, A., Conlon, M., 2015. A clustering approach to domestic electricity load profile characterisation using smart metering data. Appl. Energy 141, 190-199.

Morgado, E., Martino, L., San Millán-Castillo, R., 2023. Universal and automatic elbow detection for learning the effective number of components in model selection problems. Digit. Signal Process. 140, 104103.

Moutsinas, G., Guo, W., 2020. Node-level resilience loss in dynamic complex networks. Sci. Rep. 10 (1), 3599.

Papageorgiou, T., Michaelides, P.G., Milios, J.G., 2010. Business cycles synchronization and clustering in europe (1960-2009). J. Econ. Bus. 62 (5), 419-470.

Quah, C.-H., Crowley, P.M., 2010. Monetary integration in East Asia: A hierarchical clustering approach. Int. Finance 13 (2), 283-309.

Rajabi, A., Eskandari, M., Ghadi, M.J., Li, L., Zhang, J., Siano, P., 2020. A comparative study of clustering techniques for electrical load pattern segmentation. Renew. Sustain. Energy Rev. 120, 109628.

Rand, W.M., 1971. Objective criteria for the evaluation of clustering methods. J. Amer. Statist. Assoc. 66 (336), 846-850.

Ravn, M.O., Uhlig, H., 2002. On adjusting the hodrick-prescott filter for the frequency of observations. Rev. Econ. Stat. 84 (2), 371-376.

Roth, A.E., 1988. The Shapley Value: Essays in Honor of Lloyd S. Shapley. Cambridge University Press, Cambridge, United Kingdom.

Sachs, J., Moya, D., Giarola, S., Hawkes, A., 2019. Clustered spatially and temporally resolved global heat and cooling energy demand in the residential sector. Appl. Energy 250, 48-62.

Shapley, L.S., 1951. Notes on the n-person game – II: The value of an n-person game. Rand Corporation, United States.

Voulis, N., Warnier, M., Brazier, F.M., 2018. Understanding spatio-temporal electricity demand at different urban scales: A data-driven approach. Appl. Energy 230, 1157-1171.

Wang, X., Smith, K., Hyndman, R., 2006. Characteristic-based clustering for time series data. Data Min. Knowl. Discov. 13, 335-364.

Ward Jr., J.H., 1963. Hierarchical grouping to optimize an objective function. J. Amer. Statist. Assoc. 58 (301), 236-244.

Wu, J.-S., Chen, R.-J., 1994. An algorithm for computing the reliability of weighted-k-out-of-n systems. IEEE Trans. Reliab. 43 (2), 327-328.

Yilmaz, S., Chambers, J., Patel, M.K., 2019. Comparison of clustering approaches for domestic electricity load profile characterisation-implications for demand side management. Energy 180, 665-677.