Uncovering Sets of Maximum Dissimilarity on Random Process Data
Authors: Miguel de Carvalho
Gabriel Martos
Venue: Accepted by TMLR ()

Abstract: The comparison of local characteristics of two random processes can shed light on periods of time or space at which the processes differ the most. This paper proposes a method that learns about regions with a certain volume, where the marginal attributes of two processes are less similar. The proposed methods are devised in full generality for the setting where the data of interest are themselves stochastic processes, and thus the proposed method can be used for pointing out the regions of maximum dissimilarity with a certain volume, in the contexts of point processes, functional data, and time series. The parameter functions underlying both stochastic processes of interest are modeled via a basis representation, and Bayesian inference is conducted via an integrated nested Laplace approximation. The numerical studies validate the proposed methods, and we showcase their application with case studies on criminology, finance, and medicine.

Published in Transactions on Machine Learning Research (05/2024)
Uncovering Sets of Maximum Dissimilarity on Random Process Data
Miguel de Carvalho
miguel.decorvulho@@ed.ccc uk
School of Mathematics, University of Edinburgh, UK CIDMA, Universidade de Aveiro, Portugal
Gabriel Martos
gmarto edu
Departamento de Matemática y Estadistica, Universidad Torcuato Di Tello, Argentina
Reviewed on OpenReview: https: // openreview. met/ forum? id=ntWCJr1D08
Abstract
The comparison of local characteristics of two random processes can shed light on periods of time space at which the processes differ the most. This paper proposcs method that learns about regions with certain volume, where the marginal attributes of two processes are less similar. The proposed methods are devised in full generality for the setting where the data of interest are themselves stochastic processes, and thus the proposed method can be used for pointing out the regions of maximum dissimilarity with a certain volume, in the contexts of point processes, functional data, and time series. The parameter functions underlying both stochastic processes of interest are modcled via basis reprosentation, and Bayesian inference is conducted via integrated nested Laplace approximation. The numerical studies validate the proposed methods, and showcase their application with case studies on criminology, finance, and medicine.
1 Introduction
Everyday millions of data patterns flow around the world at unprecedented speed, thus leading to an explosion on the demand for modeling stochastic process data-such as point processes, time serics, and functional data; each of these types of data plays a key role machine learning, as can be seen, for instance, from the recent papers of Berrendero al. (2020), Faouzi & Janati (2020), and Xu al. (2020). Hand in hand with this shock on demand arrived pressing need for the development of data-intensive methods, techniques, and algorithms for learning and comparing random processes.
Problem. We deal with the following general problem on the comparison of stochastic processes:
For pair of random processes, what is the region given nolume-whert they sta- listically differ the most?
Since the target of interest consists of set that fulfils an optimization criterion i.e. period of time region over which marginal feature of two stochastic processes differ the most-some of the main concepts in this paper can be framed
optimization problem over set function. Unlike standard functions that assign number to another number, set functions assign value to each set. For example, probability measure F(A) = Pr(A), where A is an event (say, A = [0, oo)), calculates the probability of that event; another example is the counting measure F(A) = #A which simply counts the number elements in a set (for instance, A (1) As these examples illustrate, the sets of interest can be either discrete or defined on continuum. For the purposes of this paper, since the goal is identify region with given volume, will focus sets defined continuum.

---

Published in Transactions on Machine Learning Research (05/2024)
The canonical problem in set function optimization is of the type.
F(A)
A
(1)
s.t.
A E g:
where F; P(d)) R is set function, d is set, P( denotes the powerset of set a and & S P(af) is the collection of feasible sets defining the constraint. Optimization problems over set functions-sus as (1)-are are commonplace in machine learning (e.g., Krause, 2010)- For a recent review on the theory of discrete set function optimization see Wu et al. (2019). Most state of the art developments have been made on discrete or combinalorial set function oplimization, especially on the class of submodular functions (e.g., Goldengorin, 2009). As a byproduct, our paper provides one of the first steps towards continuous set function optimization here the aim will be to solve (1) when f is family of Borel subsets of a compact set T C Rd.
As it will be seen in Sontion 2, the objective set function of interest in our case will be a measure of proximity between marginal features of the pair of stochastic processes of interest, whereas the collection of feasible sets introduces the constraint on the 'size' of the feasible regions-i.c periods of time or space-over which the comparison made.
Main contributions:
1. We pioneer the study, formulation, and analysis of the learning problem of tracking down sets of maximum discrimination as described above- and formally defined in Sections 2-3.
In its most standard version, the proposed learning problem is shown to be equivalent a continuous set function optimization on a monotone modular function, under Lebesgue measure constraint. Hence, a byproduct, the paper contributes to the literature on set function optimization which is mostly focused on a discrete and combinatorial framework, with a particular emphasis mono- tonc submodular functions (c.g.. Nemhauser et al., 1978; Calinescu et al., 2011; Goldengorin, 2009; Buchbinder et al., 2017; Buchbinder & Feldman, 2018), under cardinality or matroid constraints. Little is known on contimuuus set function optimization, and thus the tools, concepts, and strategies devised herein can be of further interest elsewhere.
3. Our approach is fully general in the sense that it applies most random process data (including functional data, time series, and point processes). The functional parameters of the processes of interest (say mean functions, volatility functions, intensity functions) are modelled by composing an inverse link function with a basis function representation, and Bayesian modeling is conducted via latent Gaussian models and inference is conducted via INLA (Integrated Nested Laplace Ap- proximations) (Rue et al., 2009; 2017).
4. An extension of the proposed method applies also the context where the interest is on comparing more than one marginal fcature via multi-objective version of the set function optimization problem of interest.
Related prior work. While the learning problem introduced above is new-awe while there are novel contributions that will arise from our solution to it thethere are some recent approaches in the context of functional data analysis (Ramsay Silverman, 2002; 2006; Ferraty & Vieu, 2006; Horváth & Kokoszka, 2012) that are langentially related to it, and that are brielly reviewed below.
Pini & Vantini (2016; 2017) propose interval testing procedure for functional data that points out specific differences between functional populations. Also in the context of functional data, Berrendero et al. (2016) propose a discrelization method consisting on learning to choose a finite collection of points in the domain of a set of functions in order to improve the performance of a functional data classifier. Martos & de Carvalho (2018) propose a Mann-Whitney type of statistic for functional data to learn about the rogions at which two processes differ the most on aspects related with symmetry. Finally, Dette & Kokot (2021) develop hypothesis tests for the equivalence of functional parameters in two sample functional data setup.
2

---

Published in Transactions on Machine Learning Research (05/2024)
8
1apx0 and (ooo
40
30
By and By
and Dv
20
A0
A119/25
Time
Time
(a)
(b)
(c)
Figure 1: Sets of maximum dissimilarity for Examples 1-3. The sets depicted using black dotted segments in the case of mean functions and volatility functions (a, c), and the top of the box with black dotted line in the case of intensity functions (b). Raw data alongside the true and estimated functional parameters 0x and By are depicted in red and blue respectively; the proposed inferences for the sets will be discussed in Section 2.2.
Our approach differs from the ones mentioned above in a number of important ways. Perhaps the most important oncs that: i) here the goal is not test hypothesis but rather to learn about regions with given volume where two processes differ the most; W) our approach applies to random processes in general whereas the methodologies reviewed above have mainly been designed with the context of functional data analysis in mind.
Outline. In Section 2 we introduce sets of maximum dissimilarity, present examples, and introduce the inference methods, whereas in in Section we comment on extensions. In Section we showcase the proposed methods real data applications. Closing remarks are given in Section 5. Proofs and numerical experiments with artificial data are presented in the Appendix. Table 1 in the Appendix lists the main symbols and notation used throughout the article. We write X(t) instcad of X, when typographically convenient.
2
Learning About Sets of Maximum Dissimilarity
2.1 Sets of Maximum Dissimilarity
Instances
Prior introducing sets of maximum dissimilarity in a formal fashion, we introduce some simple instances of the concept based different parameter functions 0x and 6y. The respective sets of maximum dissimilarity and parameter functions for Examples 3 to be presented below are depicted in Fig. 1. The 'size' of these sets (c > 0) should be specified by the user based on the desired level of gramularity for the analysis; this enables tracking of maximum differences on a weekly, monthly, or annual basis, for example. This will become clearer through the examples below and the data applications in Section
Example 1 (Mean Functions). Let X, = (t) + (t)) and Y; = By (t) + (t) be two stochastic processes defined on the unit interval, = [0, 1], with
(1) F(X,)) b(() cos((10) - 0.75)
(2)
(t) = E(Y) -b(t) + +33sin (1tt).
where b(t) = 1/2cxx( -05))) a bascline curve, and ex (t) and (t) are zero mean Gaussian error functions. Fig. 1(a) shows the connected set of size 0.2 where both mean functions differ the most the 1.2 sense.
3

---

Published in Transactions on Machine Learning Research (05/2024)
Example 2 (Intensity Functions). Consider the intensity functions associated to two point processes
xx (t) Ax (t) 7 - f2)/2),
(3)
(t)= =0x (t),
defined over the region T' [-3 with 7.6 O. Fig. 1(b) shows the true ball of maximum dissimilarity with area 4t at which the two intensity functions differ the most in the I sense in the case where 7 100 and 8 = 1/2.
Example (Volatility Functions). Suppose X2 and Y, are the log returns of two stock markets, with E = E(Y) = O, and that the goal is to search for the period of about a quarter (think of T [0, 11, so that 0.25), where the volatility between both markets differed the most. Let
[x(()) = xx (t) == (var(X.)) (ly(1) - oy (1) = {var(Y,))3/2.
Fig. I(c) depicts a simulated example of two artificial stock prices evolving during a period of 1 year (T [0, IJ), along with the month period over which the volatilities in both markets differed the most in the L sense.
To allow for visualizations, in the examples above we focused on low-dimensional instances but the theory to be presented next holds general for any compact ground set 7 R
Regions of maximum dissimilarity
Let 0x = (t( and 0y = (t) be functional parameters characterizing q different marginal features of the processes X and Y for E C Rd. Throughout, we will assume that the ground set is compact and that 0x and By live in the Banach space (LP(T), llp), with I[fllp ( If I du) where μ a measure over the Borel sets on T; will refer to |] (Ja [f] du) as the L sub-norm norm over A C T. We equip the set A of compact and convex subsets of the ground set T, with the Hausdorff distance
max d(t, B), max d(t, a(t.A)) EEA
dA(A, B) = max
A.A
YEB
where d(t, B) is the minimum Euclidean distance from the point E A to the set B.
A compact and convex set A C T is said to be a region of macimum dissimilarily (in the L sense), with volume bounded by 0, if it maximizes F(A) ||0x 0y(1( and if its volume, |A], does not exceed e; the formal definition is as follows.
Definition 1 (L" Region of Maximum Dissimilarity). Suppose 0x, By E (T)T) and T C Rd is compact. A region of marimum dissimilurily (RMD) is defined as a set A C that solves,
10x 0r
max AGT
8.1.
|A| c,
(4)
A is compact and conver,
for a fized e 2 0. In addition, (2 x 0y) ) = |10x By 1 (A() is said to be the dissimilarity indez.
The optimization problem in (4) is continuous set function optimization problem similar to (1), with FAA) |10x - By 11b4) of T, F [A: |A| S c, A is compact and convex).
Clearly, F(A) is an increasing set function, that is, if A C B then F(A) S F(B). Also, F(A) is modular (or additive) in the sense that F(A) + F(B) = F(AU U B) + F(An B). While is evident that A exists for a few specific examples (say, Air = T almost everywhere, if T convex), the existence of A in general is not straightforward and is proved below in Theorem 1. RMDs and dissimilarity indices also have a number of attributes which we summarize Theorem
4

---

Published in Transactions on Machine Learning Research (05/2024)
Theorem 1. Suppose 0x, By LP()) and T C R is compact. The quantities A and D obey the following
properlies:
A erists, for every c 2 O.
2. Suppose p(A) = [AI. Then, the RMDs of (K0x, K0y) and (L0x, L0y) are respectively A = A and A = -atppe where K((tt) = a + (8(t), L0(t) = a(a + 3t), a E R, and B #+0, 0.
3. (0,, 0y) is a distance over O = A E (P(T), E Fch, for c 0. 4. (2 0y) is non-decreasing as function of c, for fixed 0 By E LTT)).
Theorem warrants some comments. The existence of RMDs (Claim 1) follows from the continuity of the volume functional, |A], and the compactness of the search domain, Fei see Appendix A. An RMD needs not however be unique, as can be easily seen by considering the limiting case 0x = By for which every compact and convex subset of T with measure is RMD. Claim 2 shows how RMDs are impacted by group of linear transformations acting either on the functional parameter over time; the assumplioos p(AA= = | and | # 0 are only required for A = AA as indeed A = A holds more generally. Claim 3 implies that dissimilarity indicos have a metric interpretation. Finally, Claim notes that D = Dr ((0xx,y)) By) cannot decrease as с increases.
For practical considerations entailed in the estimation of A from data (more details in Section 2.2), next introduce the L balls of maximum dissimilarity as suitable parametric approximation RMDs.
Balls of maximum dissimilarity
Let Bp be the family of all closed balls in LP(T), for p 2 1, that is,
A = (B(t,r) O,t
where B(t,r) {s ||s tlp S r}; to ease notation we drop the dependence of B(t,r) on p. A more parsimonious option for modeling and computing is to put more structure on the shape of RMDs, and this leads the next definition.
Definition 2 (L" Ball of Maximum Dissimilarity). Suppose 0x By [P) and T C Rd compact. An IP ball of marimum dissimilarily (BMD) is defined as set B. T that solves,
|10 By 11(
max BCT
s.t.
[B] S c
(5)
B E Bp.
for a fized c 2 0.
As in Definition refer to 9:- 2: (0x,0r) F(B:) as the dissimilarity index for BMDs. Since the volume of an (T) ball of radius in Rd is
(2r[(1/p11)] r(d/p+1)
B(t,r)|=
for all t T, where [(z) 00 --- -00 dr is the Gamma function, it follows that the volume constraint BMDs, |B] S can be rewritten function of the radius, that is
(cI(d/p 1/
S 21(1/p+11
R.-
(6)
Equation (6) is computationally convenient and will be revisited in further detail in Section 2.2.
Given the similarities between the definitions RMDs and BMDs it not surprising that they have identical properties (c.g., for 0, (=x, 0y) a also distance and Φ is non-decreasing). More generally, the following theorem holds:
5

---

Published in Transactions on Machine Learning Research (05/2024)
Theorem 2. Suppose 0x, By LP(T) and T C Rd is compoct. The quantities B and 2 obey the following properlies:
B ezists, for every c c26 0.
2. De is conlinuous, and the "argmax" correspondencee of center center-radius radius, Or, [0, 0o) - T x [0, Rel, defined 0s ac = {(t,r) E x Re]: f(t,r) = 22), , is upper hemicontinuous for every c 2 0, where =(t,r)= = F(B(t
Theorem 2 proves the existence of BMDs and notes that the smoothness of Re is inherited by D. Next, we switch gears and focus on learning about BMDs from data.
2.2 Learning About BMDs from Data
Computing and optimization
As anticipated earlier, BMDs are computationally convenient as a suitable parametric approximation RMDs. This becomes apparent from (5) as finding optimal ball corresponds to standard constrained optimization problem, where the objective function depends solcly the center-radius vector with d + components. In addition, (6) highlights another compulationa advantage of modeling RMDs via BMDs: The volume constraint can be directly expressed as a function of one of the arguments of the objective function, specifically the radius. To put differently, the key takeaway from (5) and (6) is that the set function constrained optimization problem leading to BMDs can actually formulated as a standard continuous optimization problem over [0, Re] C Rd+1, Indeed, follows from (5) and (6) that computing a BMD is equivalent to solving
(fx, (t,r) T x ReI).
(7)
where f(t,r) F(B(trr)) When the center of the optimal BMD (f²) is 'sufficiently far' from the boundary of T, the optimal radius (r²) is Re in (6). That is, when d(t", a()) > Re the optimal radius is r = Rs (as
rt,,r) a non-decreasing function of r) in which case the optimization problem in (7) resumes to searching for to e Rd. This also implies that often in practice the marginal posterior of r is essentially degenerated.
Latent Gaussian model specification
To model BMDs in applications we consider a version of the latent Gaussian model specification in Rue et al. (2009) adapted to our setup; to ease notation, below we only refer to ((t,, and denote its functional parameter by ((t) = (t), but all comments apply to Y(t) as well. latent Gaussian model is essontially a Bayesian generalized additive model that assigns Gaussian priors to parameters and a possibly non-Gaussian prior its hyperparameters. Specifically, suppose that (tt) h()(tt) is in the exponential family, with its mean function coinciding with the functional parameter, and that
0(t) 90 B.0.(1).
Here (á = ot))), a set of basis functions in LP(T), 3= 3g) is a parameter, and g is an inverse link Junction. Following Rue el al. we assign multivariate Normal prior with a sparse precision matrix (Q) to B, which induces a Gaussian process prior on g (0(()) with conditional independence property. Many functional parameters can be modeled in this way including those from Examples and all numerical instances in Section 4 and Appendix C. The theoretical developments from Section 2.1 apply however more generally beyond the modeling assumptions made over this section.
INLA- based inference for balls of maximum dissimilarity
We now discuss how conduct inference for balls of maximum dissimilarity. It is well-known that the latent Gaussian model described above can be fitted with an Integrated Nested Laplace Approximation (INLA)

---

Published in Transactions on Machine Learning Research (05/2024)
(Rue et al., 2009); the method is effective even when the dimension of the precision matrix Q is large, and is particularly tailored for the case where the number of hyperparameters, a, is moderate (say, 6- 12). INLA is a deterministic method for approximating the marginal posterior of each parameter that is based on the Laplace approximation; loosely speaking, the Laplace approximation is an approximation for integrals of the type f (=- dy for large n, that approximates the integrand (e (e="/()) with a Gaussian density centered mode and sets the covariance matrix as the inverse of the curvature (around the mode); see, for instance, Young & Smith (2005, Section 9.7). Below, we sketch some brief details on INLA. The first step of INLA approximates the marginal posterior of via the Laplace approximation, that is,
p(a data)= = p(a, B data)
p(a, 3 data) =
p( data) p(3 a, data) 8=9;
where F(B a; data) is the Gaussian approximation based on the mode of the full conditional of B, and where 3n is the mode this approximated full conditional for given a. Next, INLA approximates the marginal posterior of each component of B. Let 3_; be the elements of B, except Br. Similarly (9) it follows that
p(3 a, data) p(a, B data)
(10)
p(3-; a 8i, data)
which be approximated using Laplace approximation for p(B-i a; Br, data). Finally, the marginal posterior density p(3; data) is obtained by numerically integrating out Below, INLA is applied to the data from processes X and Y as Dx and Dy. Estimation and inference for BMDs can be conducted using Algorithm 1 that combines the deterministic nature of INLA along with sampling.
Algorithm ANLA-based posterior inference for BMDs
Step 1:
Fit the marginal posterior densities, p(3x, (xx) and p(3y, Dy), using the Integrated Nested Laplace Approximation, for i = 0,..., Bx and = 0..... By.
Step 2: Sample m posterior draws from the full posterior of Bx (3x.0....,7)) ,x, and By (By By,By,, ) given by
...., p(m) iid p(3x Dx), to generate m postcrior trajectorics from the functional parameters using (8); that is, for k I,..., m, do
8
8 ......, alm) üid p(3r (Dr)
x = Px.0 (
+
Σ
Bx
B
(2)
(0 9 (K)
Σ (t)
By
(11)
Step 3: Use the posterior trajectorics from Step to obtain a sequence of posterior BMD draws for [B(R)) B(t())) re )
with (1 re (()))) solving the optimization problem in (7), for =1,....,11
Algorithm/ warrants some comments. Step is deterministic, it follows by numerically integrating out the lyperparameters in (10), and to facilitate its implementation (Martins et al., 2013; Lindgren et al., 2015) from R (R Development Core Team, 2022), which is also equipped with routines that facilitate the implementation of Step 2 (e.g. inla. posterios. seatterior.smmlle... Step 3 boils down to solving the optimization problem in (7) using the pair of posterior trajectories in (11) thus we solve (7) per each INLA itcration and the posterior quantifics the uncertainty surrounding the optimal centre and radius. Except where mentioned otherwise, in all experiments reported below, we draw m 1 1000 times from the posterior distribution of BMDs using Algorithm 1.
recommend using the R-INLA package

---

Published in Transactions on Machine Learning Research (05/2024)
3 Multi-Objective RMDs
We now consider the context where the interest is on learning about regions over which set of marginal features of two processes most differ. Let
x (0x,1, (0x 00.0)) = (fx.,()........A (x)))))
0r = (0y,1,...,0y.)) 0y.a)= (0y, Oy (1))
be functional parameters characterizing marginal features of these processes. In this section
F((A) ||0x, fy ((A))
(12)
denotes the set objective functions of interest, for = 1.....g. 9. Ideally, we would aim simultaneously maximize all targets (Fi increased some other may be decreased. The following Pareto optimal concept induces order over the collection of feasible sets Fer and it defines optimality via compromise across all objective set functions, in the sense that improvements one target cannot be made at the cost of deteriorating another target.
Yet, in practice targets may conflict each other, that is, if one is
Definition 3 (Pareto Optimal Region of Multi-maximunt Dissimilarity). Let (((A) be defined in (12). The set A E F. is Pareto optimal region of moultimarimam dissimilarity if there crists no other set A E Fe such that P((A) 2 (.(A)), for all i E (1....,q), and P((A) > (A?),, for al least one j E {1.....g) q).
In common with standard theory for multi-objective function, in our set function context the set of Pareto optimal RMMDs can be analytically obtained only in very specific cases, and thus we need to resort to scalarization (Pardalos al., 2017, Ch. 2) to reduce multi-objective problem into a single-obeective problem. Here, we define and characterize the following linear scalarization method for our set function context
Fwl (AA),
(13)
(W1,.... W.). (0, 00)9, and we will refer to
A arg max F (A), AEF.
(14)
as the solution to the set function linear scalarization problem with weight w.
Theorem 3. Every solution to the linear scalarization problem is a Pareto optimal RMMD.
Section 4.2 illustrates how Theorem 3 can be applied in practice.
4 Empirical Section
In Appendix C' we assess the performance of the proposed tools via a Monte Carlo study. Next, we showcase real data applications.
4.1 Thefts in Buenos Aires
Buenas Aires is the most dense mep in Argentina and its crime rales are significantly higher in compar- ison to the rest of the country. In this section we illustrate how the proposed method can reveal regions of the city where nonviolent crimcs-such as burglary, pickpocketing or nonviolent thefts have changed the most, comparing the years 2019 (pre COVID-19) and 2020 (when the COVID lockdown took place during several months). The data are publicly available online in the city hall web page (https://date manutaabeenosiree.gooo and consist of point process data on the latitude and longitude where thefts occurred during 2019 (D2019) and 2020 (Dzuz0.. Here, the functional parameters of interest are the intensity functions
02019 (latitude, longitude),
22020 (latitude, longitude).

---

Published in Transactions on Machine Learning Research (05/2024)
and its BMD will represent region of the city, of a given size, where the most noteworthy changes in thefts took place. The fitted BMDs were modeled according to (8) using log- Gaussian Cox process with similar prior specification as in Appendix C. In Fig. 2(a) we depict the estimated BMD corresponding to an area of 8km² along with an heat map of the differences in the estimated posterior intensity functions between consecutive years; the value of 8km² was chosen for illustration as corresponds to about twice the size of the largest neighborhood, which is Palermo. We also depict Fig. 2(b) an heat map of the posterior density corresponding to the center of the BMD which shows that these are substantially concentrated, thus suggesting low uncertainty on the fitted BMD.
-34 56
-3465
Latitude
34 60
Latifude
33
2000
100X
2000
5003
-94.5-
-3473
-58:5
4544
-5:40
-5.35
Longitude
Longitude
(a)
(b)
2019
2020
-2.55
Laiitude
Lattlude
2000
-0.70
-38.30
-38.5
-00 A0
-3.25
50
45:45
Longitude
Longitude
(c)
(d)
Figure 2: (a) Fitted BMD corresponding c == 8km2 along with a contour of pointwise differences between fitted posterior intensity functions over consecutive years. (b) Posterior density estimation corresponding to the centers of of the fitted BMD. (c) and (d) shows the fitted intensity functions for 2019 and 2020.
To clarify the applied meaning of such BMDs provide some additional background the social context surrounding the empirical analysis. In Fig. 2(c-d) we depict the fitted intensity functions corresponding to both years. As can be seen from Fig. 2, in 2019 and 2020 thefts were more likely to occur in APRV (Almagro, Palermo, Recoleta, and Villa Crespo) which are some of the neighborhoods where several commercial and touristic activitics took place. Yet, important differences on the estimated intensity functions are perceived between both years. During the first half of year 2020, local authorities took strong social distancing measures such the limitation the access the public transportation system, restrictions business and commerce

---

Published in Transactions on Machine Learning Research (05/2024)
during the day, limitations on gatherings and tourism activities, restriction to the capacity in bars and restaurants, among others. The difference on the estimated intensity functions between consecutive years evident from Fig. 2 and the implied reduction of thefts over 2020 is in line with the findings of Mohler et al. (2020) that report similar evidence on the effect of COVID-19 lockdown and social distance policies in nonviolent crime. The BMD Fig. 2(a) suggesis that APRV (Almagro, Palermo, Recoleta, and Villa Crespo) are the neighborhoods where there was most impactful effect of COVID-19 lockdown. To put it differently, while nonviolent crime has decreased during lockdown over the entire city, what the fitted BMD highlights is that such reduction was relatively much higher in the APRV neighborhoods.
4.2 Volatility in Stock Markets
Our second illustration will shed light on the multi-objective approach from Section 3. Data were gathered from Yahoo Finance and consist of monthly values of the NYSE and NASDAQ composite indices from the New York Stock Exchange ((X+)) and the NASDAQ Stock Exchange ((Y,)), respectively. The data ranges from Jamuary 1990 to May 2021, thus covering variety of episodes of financial turbulence such as the dotcom tech bubble that peaked around 2000, the subprime crisis that started around 2007, and the recent COVID 19 global pandemic. In the multi-objectivve RMD analysis to be conducted here, we will consider two functional parameters of interest: The mean values of the indices over time and the volatility of their log returns, that is,
(t(()= = E(x,), (t = F.Y)),
(x(()= = [=(log(Xx/Xx-)))] (O) [F(log (V.og((./V..))888
These functional parameters were modeled according to (8), using an identity link function and B-spline basis functions, choosing the number of basis functions using the DIC (Deviance Information Criterion: Spiegelhalter et al. 2002; 2014), and using uninformative prior.
We consider intervals of six months, one year, and two years (corresponding to c = 6, 12, and 24) and we aim to evaluate on what periods of such length these two stock markets differed the most in terms of both average returns as well as volatility. We thus seek for the interval of time B = It - r2.ta + rč] that as in (14) maximizes the following scalarized sot function optimization problem,
max (F, (t,r) T 10, Rel),
(15)
where w is the scalarization parameter and
pitt Fu(((t,r)) = w | (mx(u)- - my (u) + (1- - w) f (ox(u)- - or (u))
+(1
It follows from Theorem that every solution to the linear scalarization problem (15) is a Pareto optimal BMMD (ball of multi-maximum dissimilarity), and hence Pareto optimal BMMDs obtained by lincar scalar- ization have the nice feature of allowing one to put more emphasis according to how we set w. That is, by setting w = 0 or w = 1, we only consider volatilities or mean values respectively and the analysis corresponds to a standard BMD, while for w E (0, 1) absolute values in differences between mean functions are more important than those in volatilities as w increases. In terms of computing we adapt Algorithm to the multi-objective context. That is, inference about. the BMMDs is conducted by sampling m = 1 000 times from the postcriors for mcans and volatilitics -nd rather than solving (7) as in Algorithum 1 we now solve the scalarized set function optimization problem in (15).
the mean values or volatilities
In Fig. 3 compare the results of the single BMD analysis [panels (a) to (d)] versus the multi-objective analysis [panels (c) and (f)) considering a value of e corresponding to a period of six months, one year, and two years. As can be seen from Fig. 3(a b), the BMD associated with the mean (w = 1) concentrates around the subprime crisis, indicating that July 2006 to July 2008 is the period over which mean levels of NYSE and NASDAQ differed the most. In Fig. 3(c-d), we see that the BMD associated with volatility (w = 0) ranges from October 1999 to October multi-objective approach with w 2/3 is depicted in Fig. 3 (e-f) and it suggests that volatility has greater
corresponds the dotcom bubble burst. Finally, the
10

---

Published in Transactions on Machine Learning Research (05/2024)
BMD for mean (10 =
10020
8x and 6y
10080
and Ol
Posterior
200
6x
5008
0.00
2000
Time
2010
1780
1561
2002
2010
2015
2300
(a)
(b)
BMD for volatility (w 0)
15000
8x and By
10000
a and
Posterior
0000
0.27
0.00
Time
1200
1006
2000
2006
2010
2015
2300
1
(c)
(d)
Multi-objective BMD for volatiilatility (w 2/3)
0.08
15000
0.08
Postenior Density
6x and By
10:00
60
0/04
6x and
0000
0.50
1002
Time
2010
1980
2005 1.
2015
2300
(e)
(f)
Figure 3: (a, c, e) Black solid segments corresponds to fitted multi-objective BMDs for six monts, one year, and two years for w = 1, 0, and w = 2/3; the charts also show the realized (solid) and estimated (dotted) volatilities corresponding to NYSE (red) and NASDAQ (blue). (b, d, f) Marginal posterior density for the centre of the BMD.
control on the objective function in (15): that is, even when we set 2/3 that is, even when we set more emphasis the differences in means rather than the differences in volatility we still get similar
11

---

Published in Transactions on Machine Learning Research (05/2024)
result setting O, as end up recovering the period of the dotcom bubble burst can be seen from Fig. 3(e-f).
4.3
Electrocardiogram Data (ECG200)
the FCG2200 dataset contributed by Olszewski (2001). The data is the result of monitoring electrical activity recorded during one heartbeat and it consists of 200 ECG signals sampled at 96 lime instants, corresponding to 133 normal heartbeats (Dx) and 67 myocardial infarction signals (Dy); the data are publicly available from the UCR Time Series Classification and Clustering website (http://www.. cs. ucr www.uuur..uaea..
For final illustration
In this illustration, the functional paramoters of interest are the mean functions of ECG signals for both classes (normal heartbeat (x = E(X,), and myocardial infarction (t) E(Y.)) and one of the goals of the analysis is to track down periods, of a given length, over cardiac cycle where the differences between the two classes is most pronounced. To model these functional parameters the Gaussian process prior specification in (8) was once more applied using an identity link, B-spline basis functions, the DIC to select the number of basis functions, and a Matérn covariance function with the PC prior of Fuglstad et al. (2019) setting P(o 1) 0.001 and P( < 0.05) 0.001.
0.00
f,-
0.15
Posterior Density
ECG Signal
2
19
38
77
96
38
67
96
Time
(a)
(b)
Figure 4: ECG200 dataset: (a) Black solid segments correspond to fitted BMDs, Bio C B C B%:: the chart also shows the raw data (dots) and fitted mean functions (dashed) along with 95% crodible bands, where red and blue respectively correspond to normal heartbeats and myocardial infarction signals. (b) Marginal posterior density for the centre of the of maximum dissimilarity Bio-
In Fig. 4(a), we depict the fitted posterior estimates for a sequence of BMDs (Bio. Bio- and Bgg) using black solid segments, along with the raw data, and the fitted mean functions with 95% credible bands. The obtained BMDs uncover periods of a given length where we observe largest differences between the estimated mean F.GG functions. Informally, we may think of such BMDs as corresponding to intervals about 10 deciscconds, since the 96 time instants cover about cardiac cycle for all subjects and thus are not expected to last longer than second. The fitted BMD centers localized around the time instants 22 55. All in all, the analysis suggests that while normal heartbeats and myocardial infarction signals have similar 'peaks' at the beginning of the sample period (i.c. they have similar Q waves, in ECG signal analysis terminology). immediately in the period right after (i.e. over their ST segments) they greatly differ.
We close the analysis with two final remarks. First, in this illustration the fitted BMDs verify the chained inclusions Bi C Bj Big but this property does not hold in general-nor for the fitted BMDs, nor for the true ones; counterexamples can be constructed either numerically or analytically. Second, although BMDs unrelated to classification, since the ECG200 dataset is popular benchmark for new classifiers it
12

---

Published in Transactions on Machine Learning Research (05/2024)
may be sensible to ask whether the accuracy of some classifiers at discriminating outcome classes (discased- nondiseased) can be improved by focusing on BMDs rather than treating the entire time horizon equally; we leave such open problem for future analysis.
5
Discussion
Sets of maximum dissimilarity and their variants are here proposed tool for acquiring knowledge on the region with a given size where two stochastic processes differ the most. The proposed learning problem is shown to be equivalent to a continuous set function optimization on a monotone modular function, under a Lebesgue measure constraint. The cxistence of the proposed sets of maximum dissimilarity is nontrivial but we prove their existence, and illustrate with artificial and real data that it only requires a moderate computational investment to learn them from data. The proposed methods are developed in full generality for the setting where the data of interest are themselves stochastic processes, and thus the proposed toolbox can be used for unveiling the regions of maximum dissimilarity with given volume, for variety of random process data. All modeling was framed within a latent Gaussian framework, with inference being conducted using the Integrated Nested Laplace Approximation; clearly, other computational approaches could have been employed as well, including, for example, variat ional inference (Blei et al., 2017). A multi-objective version of the proposed framework also devised to learn about multi-objective RMDs, where several functional parameters are cans characterizing specific fcature of the processes being compared.
While the theoretical developments from Section 2.1 establish the existence of general compact and convex sets of maximun dissimilarity, BMDs turn out be much convenient simplification for variety of reasons. First, numerical optimization is much more challenging with general RMDs, whercas for BMDs it is relatively simple as can be seen from (7). Second, the inference for general RMDs would entail averaging posterior simulated RMDs that is, averaging sets sets-whereas for BMDs it suffices to compute the posterior mean of the (d + 1)-dimensia centre-radius pair.
Some final comments on future research are in order. Firstly, while the paper focuses on scenarios where the sets c, this approach could be integrated into decision theory framework; although it remains open problem for future analysis, Appendix E provides hint on how this issue could be addressed. Secondly, this paper pioneers and introduces sets of maximum dissimilarity, which have been designed to highlight potential differences. Yet, in cases where {x,) = (Yr), they may identify spurious regions. Assessing the statistical significance of these regions through multiple testing procodures seems conccivable, but the effectivencss of these approaches
this context requires further theoretical scrutiny and numerical analyses. Finally, discrete analogues of our problem may also be of interest in their own right, and may present themselves as natural option for devising other practical algorithms for approximating continua RMD.
Acknowledgments
We are grateful to the Action Editor and three anonymous reviewers for their suggestions and construc- tive feedback that have significantly improved the manuscript. We thank Vanda Inácio de Carvalho for comments and feedback and Finn Lindgren for discussions on INLA. MdC was partially supported by the Royal Society of Edinburgh and by FCT (Fundação para a Ciência a Tecnologia, Portugal) under Grants https://doi.org110.554.4. 10.54499/ BUBB99/UUBB///OO HDDD0//441///100 and http:////.rrrr/0000/////////
References
Milton Abramowitz and Irene A Stegun. Handbook of Matheratical Functions. Dover, New York, 1964.
CD Aliprantis and KC Border. Infinite Dimensional Analysis. Springer, New York, 2006.
Yoav Benyamini. Applications of the universal surjectivity of the Cantor set. The American Mathematical Monthly, 105(9):832-88) 1998.
José R Berrendero, Antonio Cuevas, and José Torrecilla. Variable selection in functional data classification: A maxima-hunting proposal. Statistica Sinica, pp. 1199-63, 2016.
13

---

Published in Transactions on Machine Learning Research (05/2024)
José R Berrendero, Beatriz Bueno- and Antonio Cuevas. On Mahalanobis distance in functional settings. Journal of Machine Learning Research, 21 (9):1-33, 2020.
David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: A review for statisticians.
Journal of the Americon Stalistical Association, 112(518):859 877, 2017.
Hans W. Borchers, praema: Practical Numerical Math Functions, 2021. R. package version 2.3.6.
Niv Buchbinder and Moran Feldman. Submodular functions maximization problems. In Handbook of Ap- prorimation Algorithms and Metaheuristics, 2nd ed, pp. 753-T7 183-788.4 Chapman and Hall/CRC, Boca Raton, FL, 2018.
Niv Buchbinder, Moran Feldman, and Roy Schwartz. Comparing apples and oranges: Query trade-off in submodular maximization. Mathematics of Operations Research, 42(2):308-329, 2017.
Gruia Calinescu, Chandra Chekuri, Martin Pal, and Jan Vondrák. Maximizing a monotone submodular function subject to matroid constraint. SIAM Journal on Computing, 40(6):1740-1766, 2011.
Holger Dette and Kevin Kokot. Bio-equivalence tests in functional data by maximum deviation. Biometrika, 108(4):895-99) 993,, 2021. ISSN 0006-3444. doi: 10.1093/bionn bit//ooo//taaaaaae
biomet/asaa096. aasaa996.
Johann Faouzi and Hicham Janati. pyts: A python package for time series classification. Journal of Machine Learning Research, 21(46):1-6, 2020.
Frédéric Ferraty and Philippe Vieu. Nonpurametric Funclional Dala Analysis: Theory and Praclice. Springer, New York, 2006.
Geir-Arne Fuglstad, Daniel Simpson, Finn Lindgren, and Håvard Rue. Constructing priors that penalize the complexity of Gaussian random fields. Journal of the American Stotistical Association, 114(525):445 442-452, 2019.
Boris Goldengorin. Maximization of submodular functions: Theory and enumeration algorithms. European Journal of Operational Research, 198(1):102-112, 2009.
Lajos Horváth and Piotr Kokoszka. Inference for Functional Data with Applications. Springer, New York, 2012.
Andreas Krause. SFO: A toolbox for submodular function optimization. Journal of Machine Learning Research, 11(38):1141-1144, 2010.
Finn Lindgren, Håvard Rue, et al. Bayesian spatial modelling with R-INLA. Journal of Statistical Software, 63(19):1-25, 2015.
Thiago G Martins, Daniel Simpson, Finn Lindgren, and Håvard Rue. Bayesian computing with INLA: New features. Compulational Statistics & Data Analysis, 67:68 83, 2013.
G. Martos and M. de Carvalho. Discrimination surfaces with application to region-specific brain asymmetry analysis. Stafistics in Medicine, 11(37)1189--187 2018.
George Mohler, Andrea I, Bertozzi, Jeremy Carter, Martin B Short, Daniel Sledge, George F. Tita, Craig D Uchida, and P Jeffrey Brantingham. Impact of social distancing during COVID-19 pandemic on crime in Los Angeles and Indianapolis. Journal of Criminal Justice, 68:101692, 2020.
George L Nemhauser, Laurence A Wolscy, and Marshall L Fisher. An analysis of approximations for maxi- mizing submodular set functions I. Mathematical Programming, 14(1):265 294, 1978.
Robert T Olszewski. Generalized feature extraction for structural pattern recognition in timo-series data. Technical report, Carnegie-Mellon University, School of Computer Science, 2001.
Panos M Pardalos, Antanas Žilinskas, and Julius Žilinskas. Non-conver Multi-objectivv Optimization.
Springer, New York, 2017.
14

---

Published in Transactions on Machine Learning Research (05/2024)
Alessia Pini and Simone Vantini. The interval testing procedure: A general framework for inference in functional data analysis. Biometrics, 72(3):835-80) 845, 2016.
Alessia Pini and Simone Vantini. Interval wise testing for functional data. Journal of Nonparametric Statis- tics, 29(2):407-4244 2017.
R Development Core Team. R: A Language and Environment for Statistical Computing. R. Foundation for Statistical Computing, Vienna, Austria, 2022.
James O. Ramsay and B. W. Silverman. Functional Data Analysis. Springer, New York, 2006.
James O Ramsay and Bernard W Silverman. Applied Functional Data Analysis: Methods and Case Studies.
Springer, New York, 2002.
Håvard Rue, Sara Martino, and Nicolas Chopin. Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. Journal of the Royal Statistical Society: Series B (Stalislical Melhodology), (1)2):319 392, 2009.
Håvard Rue, Andrea Riebler, Sigrunn H Sørbye, Janine B Illian, Daniel P Simpson, and Finn K Lindgren. Bayesian computing with INTA: A review. Annual Review of Statistics and Its Application, 4:395 421, 2017.
Rolf Schneider. Convez Bodies: The Brunn Maaun--Minkwssks Theory. Cambridge University Press, Cambridge, 2014.
Rolf Schneider and Wolfgang Weil. Stochastic and Integral Geometry. Springer, New York, 2008.
Daniel Simpson, Janine Baerbel Illian, Finn Lindgren, Sigrunn H Sørbye, and Havard Rue. Going off grid: Computationally efficient inference for log- nag-Gassiian Cox processes. Biometrika, 103(1):49-70, 2016.
David J Spiegelhalter, Nicola G Best, Bradley P Carlin, and Angelika Van Der Linde. Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64 (4):583-63)) 2002.
David J Spiegelhalter, Nicola G Best, Bradley Carlin, and Angelika. Van der Linde. The deviance infor- mation criterion: 12 years Journal of the Royal Statistical Society: Series B (Statistical Methodology). 76(3):485-4933 2014.
Terence Tao. Analysis Springer, New York, 2006.
Stefan Waldmann. Topology: An Introduction. Springer, New York, 2014.
Xiaofeng Wang, Yuryan Yue, and Julian Faraway. Bayesian Regression Modeling with INLA. Chapman and Hall/CRC, Boca Raton, FL, 2018.
Wei-Li Wu, Zhao Zhang, and Ding- Zhu Du. Set function optimization. Journal of the Operations Research Society of China, 7(2):183-193, 2019.
Ganggang Xu, Ming Wang, Jiangze Bian, Hui Huang, Timothy R. Burch, Sandro C. Andrade, Jingfei Zhang, and Yongtao Guan. eerim learning of structured temporal point processes. Journal of Machine Learning Research, 21(192):1-39, 2020.
G. A Young and Richard L Smith. Essentials of Slatistical Inference. Cambridge University Press, Cam- bridge, UK, 2005.
15

---

Published in Transactions on Machine Learning Research (05/2024)
A Auxiliary Lemmata
In this section we state some auxiliary results that will be used to prove the main results of this paper. Lemma 1 (Blaschke selection theorem). The set of all compuct convex subsets of fixed compact subset of Rd compact under the Hausdorff metric.
Lemma 2 (Berge's maximum theorem). Let spondence between topological spaces with nonempty compact Gr Gr + IR is conlinuous. Define the "volue function" v X R by
X
Y be continuous corre- values, and suppose that
v(z) = max{g(2, u) y p(x))
and the correspondence a: X of marimizers by
a(x) = (y e ((x) g(z, v) v(x)}-
Then:
1. The value function is continuous.
2. The "argmax" correspondence has nonempty compact values.
If Y is Hausdorff, then the "argmax" correspondence a upper hemicontinuous.
Lemma 3 (Product of correspondences theorem). The product of correspondences obeys the following prop- erties:
The product of family of upper hemicontinuous correspondences with compact values is upper hemiconlinuous with compact values.
The product of finite family of lower hemicontinuous correspondences is lower hemicontinuous.
Lemma 1 is a celebrated result on convex analysis; the version above can be found, for instance, in Benyamini (1998). Lemmas 2 and 3 are well-known results on optimization of set-valued functions; the versions above can be found Aliprantis & Border (2006, Theorems 17.31 and 17.28).
B Proofs of Main Results
B.1 Proof of Theorem 1
Claim 1. We start by showing that
= A: ||Al
is compact, for every 2 where A is the family of compact and convex subsets of the ground set T. Recall that by the Blaschke selection theorem (Lemma 1), A is compact under the Hausdorff metric. Further, since the volume functional | is contimuou (Schneider, 2014, Theorem 1.8.16), and given that F. is the preimage of the closed set [0, c). it follows that Fe is a closed subset A, and hence is compact.
Observe next that maximizing F(A) 10x (l) is equivalent to maximizing (P(A), for p 0, and we show next (PA) is upper semicontinuous under the Hausdorff metric, for every A F.. Let A, + A in (Fe,du), for a fixed e 2 0. It can be easily shown that (e.g., Schneider & Weil, 2008, Theorem 12.3.6)
(a(t) 2 sup La, (t), t
(16)
Combining (16) with Fatou's lemma yields that
(*(A) =
1a(t) - (t) p(dt)
f lim sup LA. (t)10x (t)P p(dt)
2 lim sup / (A)(0) (Y)P p(dt)
= lim sup (A,,))
16

---

Published in Transactions on Machine Learning Research (05/2024)
thus showing that (P) is upper semicontintous under the Hausdorff metric, for every A E Fc The final step of the proof is tantamount to a standard argument used for proving Weierstrass theorem. Tet sup [ (A): A E { By definition, for every e there exists a maximizing sequence A, E F. such that (An) - Me- By compactness, we can assume that A, + A Upper semicontinuity of F*(A) implies that Ue = lim sup, (An) S (P())), and on the other hand we have We 2 F(AA) since ue is the supremum. This proves that ue = (P() the maximum of FA(A), subject to A Fer and hence A exists that solves (4).
Claim 2. First, note that
|K0x Korll = ||a + 30x (ax + 30v) = 13| x 10x 0r 11,
from where it follows that set that maximizes ||0 - K0r 16:1) also maximizes 0yllb
||0x 0r ()); that is, A A for all E (0, Second, it follows from the change of variables
formula that
|10x -0001 [/ (0x(a Bt) 0r(a+ 30)3
1/8
3// [, 10x(a)) (x(w))u((( 1/D
where A= {a + ;t: E A4). Thus,
arg ( [10xx - Tor 1b49), A {.) = arg (10x 0rl]( : FatBc),
and hence, A = = A
Claim 3. Note first that D 2 0, for all < c < co. Also, holds that D. = 0 if and only if 0x = 0y in O.i indeed, D. 0 implies that 0 |10x 0r 11 > ||0x Oy l1b for every A E 3o, which is only possible if xx= = By in O,, as c > 0. Finally, the triangle incquality for || 11 and [|] = max [F()) yiclds that
(0x, 0z)
max{ 02116 A E Fc)
max - 0y) + (0y - 02 02)1l(4) AA Fc
max {100x - 01 + 100r -0))1(() A E AEF)
max{|l0x 0r :A FoJ + max{] 0z)]((a) A F.)
12(0x, (3
hence concluding the proof.
Claim 4. First, we note that an increase in e represents augmenting the search domain as
F. C Fr. for > a.
This combined with the fact that the set objective function F(A) = 110x - -0y non-decreasing (A C B implics F(A) < F(B)) yields that Dj 2 D for b b>0 a.
B.2 Proof of Theorem 2
Claim 1. Tikhonov's 's theorem implies that from the Cartesian product of two compact sets results compact set (Waldmann, 2014, Theorem 5.3.1). Hence, a consequence of this, the search domain T x [0, R.] is compact for every e 2 O. Next, it is routine exercise to prove that f(t,r) (1,r) T (0, RR] as both T and [o, R.] are compact. The final result then follows from Weierstrass theorem.
= = rt), is continuous for all
Claim 2. Let 0x,0y By LT) be fixed and set Φ;- (2 (Temma 2) which, as can be seen from Appendix A, claims that value function is continuous provided
0y). The proof uses Berge's maximum theorem
17

---

Published in Transactions on Machine Learning Research (05/2024)
that both the objective function and the constraint correspondence are continuous. In our setup, the value function is
2: max(g(c,t,r) (t,r) E p(()) max(f((tr) (t,r) E p(c)),
where f(t,r) F(B(trr) and the constraint correspondence is p [0, oo) - Rd+1 defined by
(c)) = [0, Rel,
(17)
Thus, we just have to prove that (cc) in (17) is continuous, given that the objective function g(e,t, (t, F (tt(t,r) r)} is trivially continuous for all (t. x [0, Re]. Contimuity of <p(c) follows immediately
from the product of correspondences theorem (Lemma 3), which yields that p(c) 4p xc) is continuous
as both (c) T and p2(()) [0, R.] are compacted-valueky and R. is continuous function for every 2 0. Finally, upper semicontinuity of the argmax correspondence,
a = ((t,r) T 10, Re): ,(c,t,r) = 92) ((t,r) Rdl f(t,r) V.).
follows also from Berge's maximum theorem as R d+1 is Hausdorff.
B.3 Proof of Theorem 3
Suppose by contradiction that A'c.c is the solution to the set function linear scalarization problem (14), for a fixed w E (0, 00), but that A was not a Pareto optimal RMMD; then, there would exist Pareto improvement A Fe, with A * Al Alwee so that Fi(A) 2 (,,,,,,),, for all i, and F((A) > F.(A(,,), for at least i. But then,
Σ (FF((A))
Σ w,Fi(A'
which is a contradiction as A/o.c solves the function linear scalarization problem.
C
Monte Carlo Simulation Study
Data generating processes and simulation settings
We consider the following simulation scenarios based on Examples 1 1-2 from Section 2.1:
Scenario 1 (Mean Functions) BMDs between mean functions as in Example 1. Here, X, and Y, are Gaussian processes with mean functions given in (2) and where the same Matérn covariance function is assumed for both processes. Specifically cov(X,,X,) cov(Y., = Cllss E||2, o, f), where
Culd, 0² T(u) (V202) Kr (VZrz).
2'
for (s, [0, 1J3, where K2 is the modified Bessel function (Abramowitz & Stegun, 1964, Section 9.6), and where v,v, and é are positive parameters, here set as = i f=1 1. The simulated data then a discretized version of simulated Gaussian processes evaluated over grid on the unit interval,
Dx [Xes: te (J-
with n E (10,50, 50, 100, 200} and J E {10,20) 20); the same comments apply to Dr
Scenario (Intensity Functions) BMDs between intensity functions as Example 2. Here, points drawn from non-homogeneous bivariate Poisson process with mean measures,
(HNx(AA) = rexp((( 22) dt,
(18)
FFNy (A)))
18

---

Published in Transactions on Machine Learning Research (05/2024)
for A ACT= - 3,, While the sample sizes in this scenario random quantities, given by Nx = Nx (T) and Ny = (T), the mean number of simulated points over T is (E(Nx), E(Nv)) x (2myy 2 simple yet accurate approximation that follows immediately from multiple Gaussian integrals. The simulated data are given by the following collection of points
Dx = {(X111 X2.1.... (X1,Nx X2,Nx) )
with y E (25,50) 50) and 5 E (2,4,8, 8, 16), and where Dy is analogously defined.
Scenario 1
8
: = 0.05
r 0.1
Posterior Density
6x and 0x
Time (a)
(b)
Scenario 2
By and By
0000
2
(d)
Figure 5: One shot experiments for Scenarios and (a) Black solid segments corresponds to fitted BMDs based on the simulated Gaussian process data; the chart also depicts the fitted functional parameters (dashed) along with 95% credible bands, and the true parameters (solid). (b) Marginal posterior density for the center fo plotted against the true center (rug). (c) and (d) are identical to (a) and (b), respectively, but for the simulated point process data from Scenario 2.
Prior specification and posterior inference
Inferences for the BMDs were carried out by sampling Scenarios 1 and 2, respectively. As can be seen from Algorithm 1, inferences for BMDs are constructed from the functional paramcters, and thus we now comment on what versions of (8) have been used for fitting the latter. For Scenario 1 the identily link function was used in (8) along with B-spline basis, and the mumber of basis functions was selected using the DIC. The default uninformative priors of R-INLA have been used, which consist of diffuse priors for the f's-i.c. 30 ~ N(0, oo) and 3 N(00 1000) 1000) and a long-tailer prior for the variance of the error term-iie log gamma distribution, where the gamma distribution has mean a/b and variance a/b?, with and b 10-5; see Wang et al. (2018, Section 5.2.1) for further
1000 and 500 times using Algorithm 1 for
19

---

Published in Transactions on Machine Learning Research (05/2024)
details. For Scenario 2 we follow Simpson et al. (2016) and specify Log-Gaussian Cox process using (8) by setting log link function, that links the intensity function with Matérn random field using piecewise linear basis functions over a mesh, and where the I's are Gausian-distriuuted. For the parameters of the Matérn covariance function we use the PC prior approach of Fuglstad et al. (2019) setting P(o > 1) = 0.001 and p 0.05) 0.001.
One shot experiments
We first illustrate the methods on a single run experiment for some instances of Scenarios 1 and 2. Fig. 5(a) we show the fitted BMDs for Scenario 1, along with the corresponding mean functions, on a one shot experiment with 10 and J 10, for 0.1 and 0.2. As can can be seen from Fig. 5(a), the fitted BMDs accurately recover the true B 10.165, 0.265] and Bo.2 [0.115, 0.315]. In Fig. 5(b) also display the marginal posterior density for the optimal center ty which quantifies the uncertainty surrounding the true. The marginal posterior for the radius is essentially degenerated, predicted earlier in the comments surrounding (7), and hence not shown.
In Fig. 5(c) we depict the fitted BMD for Scenario 2, on a one shot experiment with 7 = 25 and 2, for c = 2m, and display the fitted intensity functions. As is evident from Fig. 5(c), the fitted BMD nicely uncovers the true, and indeed it completely overlaps to the point that the true (depicted in gray) is barely visible.
Scenario
Scenario 2
0:15
0.10
0.00
Figure 6: Side-by-side boxplots of GHE in (19) for Monte Carlo simulation study.
Monte Carlo evidence
We redo the previous one shot analyses M 000 times, considering different samples sizes, and relying on the GHE (Posterior Mcan Global Hausdorff Error),
GHE GHE= E { [T]
1 D:) B:)de Dx Dx, Dr
(19)
to quantify how accurate on average are the estimated BMDs, B;, over 0 < c < [T). In Scenario we use DH(B2, D:) max(l(tū, -r?) (aa [[ +r2) At. + while in Scenario 2 numerical approximation of Du(B", B.)) implemented using Borchers (2021). Finally, the GHE for each simulated dataset is computed as
GHE, =
Σ
Du(B B-bwll de,
1=1
20

---

Published in Transactions on Machine Learning Research (05/2024)
where Be is the ith posterior sampled BMD, based the jth simulated sample, for 1,..., ,, y and J, and ó increases. Such behavior
= L. M. As can be seen from Fig. GHE tends to decrease confirms the expected frequentist behavior of the methods, as n and J dictates the amount of simulated data for Scenario 1, and and does the same for Scenario 2. To put it differently, since larger values of 7 and imply larger sample sizes, the observed reduction in GHE as a function of the latter parameters suggests a sensible asymptotic performance of the proposed Bayesian inferences.
D Selected Comments on the Choice of
D.1 The Coincidence Set Criterion
Optimal data-driven selection of requires criterion or loss function and remains an open problem for future analysis. Motivated by reviewer's comment, this section offers hints on potential starting points.
For T = [a b] R, practical approach is center the parameter functions and establish criterion based on the width of intervals defined by their coincidence psint(s); we term this the coincidence set criterion and will detail it next. While parts of the construction can be readily extended beyond the real line, fully developing the criterion in Rd are outside the scope of this section.
Formally, define the demeaned parameter functions
(x) ((x))0x((0 0x(() a [' 0x(u) du,
1 (1( Br
/' By (u( du,
(20)
and the coincidence set as
I= (t: (x = (t))
(21)
In words, IT is the set of points where 0x and dy coincide.
Proposition 1. Assume 0x and by are continuous functions on [a and differentiable on (a, b). Then, the coincidence set is nonempty, that is, II 0.
Proof. The proof follows from the mean value theorem (e.g. Tao, 2006, Corollary 10.2.9). First, note that
(1: 0x (1) - (v( = 0}
t (t) - (Oy b bba ( 0x(u) (y( du 0f
1
8(t)) A(b) -A(a) ) b-a
where 8(L) (1) - 0r (1) and A(b) - (aa) 0 8(u) du. The mean value theorem implies that there exists
an b), such that
((c) = (b) Aaa)
b
from where the final result follows.
To streamline the presentation of ideas, we first consider the case where (x()) and 0y touch at only a finite number of points, i.e., where has Lebesgue measure In that case, the criterion can be written as
Csc = max(Tr+1 T.
(22)
where 00 is the cardinality of and a To < < ... Fm < Fm+1 with 71 T..... Ti denoting the ordered elements of TT.
21

---

Published in Transactions on Machine Learning Research (05/2024)
The extension of (22) to the general case case-that takes into account parameter functions could coincide over set IT with positive measure and at an infinite number of points is similar but slightly more technical. First, it is convenient to partition the coincidence set as follows
II = HI UII
where 1h {r is discrete set (with the same interpretation as above, but with M I oo) and IT 1/ Ag) is collection of J S open sets. The general definition of the coincidence criterion is
Csc = max {1 a, sup(fit1 - Tol 1- lim Ti
(23)
M
where [ are the ordered clements of Hy and M & oc. Note that if 0x (t) and (t) touch at only a finite number of points, then Io and hence (23) becomes (22) with m.
D.2 Data Illustrations
Fig. illustrates the BMDs obtained using this experimental criterion with two real data examples. The findings are comparable to those presented the main paper, although the regions are slightly larger.
Electrocardiogram data
Stock market data
15000
0x and By
xx and By
10000
6000
19
38
57
77
96
1990
2000
2010
2020
Time
Time
Figure 7: BMDs corresponding to the data driven e obtained via the coincidence set approach.
E
List of Symbols
Table lists symbols and notation used throughout the article.
22

---

Published in Transactions on Machine Learning Research (05/2024)
Symbol
Description
X, Y
stochastic processes under comparison, that. is, X {Xe} and Y (8)
T
ground, or index, set over which processes X and Y are defined functional paramcters for and data on proceses X and Y Le sub-norm over A
0x, By
Dx, Dy
II (4))
F()
objective function - 0l
f(t,r)
set objective function evaluated at closed ball rtt, that is, F(B(t,r))
P.
set objective functions in multi-objective context set of compact and convex subsets of the ground set T volume functional, i.e., Lebesgue measure of A in Rd collection of feasible sets
A
A
Fe
A
region of maximum or multi-maximum dissimilarity dissimilarity index for region of maximum dissimilarity ball of maximum multi-maximum dissimilarily dissimilarity index for ball of maximum dissimilarity center and radius of ball of maximum dissimilarity set of all closed balls in L Hausdorf distance between sets and minimum Euclidean distance between a point I and set boundary of A
D.
B
Bp
du(A, B)
d(x, B)
0A
Table 1: Symbols and notation used throughout the article.
23


Reviews:
[{"review_id": "baqHDSkUDj", "reviewer": "Reviewer_BR9h", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: the paper is well written, the presentation is clear, the idea is interesting.\nWeakness: the choice of $c$, the volume of the dissimilarity set needs to be given, which is difficult in practice.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a new dissimilarity measure to identify a region where two stochastic processes differs most. The paper is well written, the proposed idea is interesting and can be useful in practice.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "In my opinion, this paper essentially provides a new graphical visualization tool to identify a region where two stochastic processes are most different, similar to a box plot or a functional box plot. I can imagine that such a tool can be useful for many researchers who work with stochastic processes. Therefore, my overall opinion on the proposal is positive. The only concern I have is that the volume of the set (i.e., $c$) needs to be given in advance, which is unrealistic in practice. A practical guideline on how $c$ should be chosen in practice should be provided. I am curious about the following questions.\n\n1. One can imagine that even if two processes are generated from the same model, one can still identify such a region with maximal dissimilarity, and how would such a region be different from two processes that have very different data-generating mechanisms? \n\n2. If there is no difference between two stochastic processes, what would be the distribution of calculated dissimilarity measures? (with a given $c$ or as a function of $c$?)", "broader_impact_concerns": "No concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "IecRrPOYT2", "reviewer": "Reviewer_epMy", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The paper introduces a novel concept of sets of maximum dissimilarity on\n   random process data, which can be applied to various types of data, including\n   time series, functional data, and point processes.\n2. The proposed method is formulated as a continuous set function optimization\n   while existing work in this area has focused on discrete or combinatorial\n   sets. \n\nWeaknesses:\n\n1. The paper lacks a detailed discussion of the computational complexity of the\n   proposed method, especially for large-scale data sets. Usually, a continuous\n   optimization problem is much easier to solve than a discrete one, as the\n   latter is often NP-hard.\n2. Some notations and expressions needs further details. For example, below (1),\n   what do you mean by 2 to the power of a set? What are the differences and\n   connections between regions of maximum dissimilarity and balls of maximum\n   dissimilarity? Why only consider learning about balls of maximum\n   dissimilarity from Data?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper considers analyzing random process data (like time series, functional\ndata, and point processes) to identify regions where two processes differ the\nmost statistically. The authors formulate the problem of finding the regions of\nmaximum dissimilarity and balls of maximum dissimilarity as a continuous set\nfunction optimization problem with a volume constraint. Theoretical properties\nof RMD and BMD are studied and multiple real data exams are provided.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Requested Changes*\n\n1. Provide more details on the notations.\n2. Make the figures more readable on gray scale. Some figures such as figures\n   1(a), 1(c), and figure 3 are difficult to read when printed on papers.\n3. Discuss the challenges and/or differences for working with continuous set\n   optimizations as compared with discrete sets.\n4. Give some insights on the connections between regions and balls.", "broader_impact_concerns": "I don't see any concerns on the ethical implications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "lGenpckp1s", "reviewer": "Reviewer_kWQx", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall, the paper is an elegant combination of rather involved theory and highly practical demonstrations. I personally do not have sufficient background for ensuring the correctness of the theoretical properties, but the problem formulation itself is novel and clearly useful and the empirical demonstrations show the method works. These are alone sufficient for a good paper.\n\nMain strengths:\n- The problem formulation itself is clear and Figure 1 supports understanding the setup early on, even if not following the mathematical basis in Introduction well.\n- The empirical experiments are extremely nice, showcasing the diversity of the possible applications this could be used for. They are simple enough so that everyone can understand the point, but still real examples where a practitioner could actually be interested in the result and was missing practical tools before this. I can see many spatial statisticians getting interested in Section 4.1 and there are also other signal processing applications like Section 4.3 where this method could be used e.g. for feature selection (as already hinted by the authors).\n- The work is in general exemplary in how it combines theory and practice, covering the whole range from mapping an interesting problem formulation to abstract mathematical concepts and then making it very concrete in terms of an easy-to-follow algorithm and clear empirical results. I will most likely be using this as educational example in some occasions.\n- Using Bayesian inference (rather than just finding the most likely ball -- I presume you could do that just as well?) is a nice additional touch which further increases the potential impact.\n\nMinor weaknesses:\n- The notion of set functions in Introduction remains a bit too vague. Maybe the Introduction would work better if you first discussed the practical examples a bit, explaining the concept of the region a bit more formally etc, and only then made the connection to set functions? Then it would be easier to make concrete what $A$, $\\mathcal{A}$ etc are in practice in some example optimization tasks? I still struggle to make the connection between the regions and the set functions clear even after reading through the whole paper.\n- Section 2.2 is a bit condensed. I believe a lot of readers would benefit if the information around Eq (7) was expanded. You now state fairly fundamental properties (\"can actually be written as a standard continuous optimization problem\") but do not really show it. I believe this is not a novel result of your work, so simply referring to other papers that discuss this equivalence in more detail would already help, and you could also spend a few more lines providing the intuition and importance for readers who are not so familiar with the connection. You could also motivate a bit more on why you choose to consider the posterior instead of simply solving (7).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors introduce a new problem formulation of discovering the region where two stochastic processes differ the most. They then map it to a canonical problem in set function optimization and prove that a solution exists. For a special case where the regions are defined as closed balls, the authors propose a practical Bayesian algorithm for finding the solution, building on INLA.\n\nThe otherwise highly theoretical paper is grounded in reality by three very clear empirical experiments that both make the concept clear and demonstrate the computational method works well.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Some recommendations on improving the writing were made when listing the weaknesses, but other than that I do not have any requests.\n\nOne visual aspect that you could think about: In Figure 2 (a) the set is communicated very nicely as visually apparent circle with thick borders, but for all of the univariate cases (Fig 1 (a), (c); Fig 3; Fig 4) the intervals are a bit difficult to see. This was particularly problematic in Fig 1 that uses gray color instead of black and I only noticed the invervals at a rather late stage. You should at least make the intervals black in Fig 1, but more generally could consider making the intervals more visible. Even though the intervals relates to the x-axis, is does not need to be visually drawn only on top of the axis.", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers generally agreed that the paper is a solid piece of work with no obvious limitations. Claims are reasonable and supported by evidence, and the resulting applications in visualizing discrepancies in stochastic processes will be of interest to the community.
Decision: Accept (poster)